MattScott

District servers are all located in the Netherlands but in a different (better) data center than we were hosting in before. We can run multiple districts per physical server. The new architecture was designed to allow us to add new district servers in different locations in the future (similar to what we're doing with LA and NY for Jericho).

Yes. The hope was that we could patch late last night. That was pushed to this morning due to a licensing issue. Everything is cleared up now.

Hi all, We will be taking Europe worlds offline starting at 10pm UTC (2pm Pacific). The plan is to take XB1 offline first so we can swap the districts to local Europe servers. That world has the lowest population, so it's easier for us to work the kinks out. We will test XB1, then bring that world online, and then move to the next. We anticipate it will take 1 hour per world to swap districts. I know this maintenance is being done on a Friday at prime time, but the European players have waited long enough, and I want to get these servers online before the weekend. Thanks, Matt

Hi all, We have 6 new servers on the West Coast and 7 new servers in Europe coming online today. That's still only a fraction of the total servers that will eventually be in place, but it's enough to host the current configuration locally in Europe with the possible exception of Social districts which will stay on the East Coast. I'll be posting the maintenance announcement soon. The plan is to stop one world at a time, change out districts, do some testing, and then open that world to players. So there will be rolling availability during the down time. Thanks, Matt

Hi there, We are including the Elf Ears in the Christmas bundles coming out on the 15th. Thanks, Matt

Hi all, A portion of the EU servers were turned over to my team last night. We are getting game server software and testing done immediately. There are enough servers that we should be able to move all action districts and fight club districts locally to Europe. The plan will be to do a short maintenance on Citadel on Thursday night to take these live. The remaining servers should all be racked early next week. I'll be watching the tracking numbers and providing status as I get it. Thanks, Matt

Hi all, We're going to be adding the Christmas items to ARMAS on January 15th during our scheduled maintenance. As part of that transition, we'll be removing the Halloween 2019 items. Specifically the following packages will be removed: Halloween 2019 - All Items bundle: https://www.gamersfirst.com/marketplace/ingame/product_details.php?storetype=g1c&gameID=20&productId=6359 Halloween Classics bundle: https://www.gamersfirst.com/marketplace/ingame/product_details.php?storetype=g1c&gameID=20&productId=6357 Halloween 2019 In-Game Rewards bundle: https://www.gamersfirst.com/marketplace/ingame/product_details.php?storetype=g1c&gameID=20&productId=6361 Get them now while you still can! Thanks, Matt

NGL, I laughed out loud when I watched this. Well done.

Typo. Unexpected.

Hi all, Lots of good progress on getting the remaining servers up and running in our new LA datacenter, however the EU servers continue to be a black box. I hate excuses, so I’m not going to make any. I haven’t had a direct response about the EU servers since they said they would deliver access on Friday. We scheduled our network team to work the weekend, but nothing has been turned over to us yet. I believe the group we are working with outsourced their EU support, so they are likely waiting on someone else, who is probably waiting on actual datacenter staff to move around hardware. With all these problems, many players may be wondering why we chose this new provider. Simply put, they are experts in defending DDoS attacks, and the end of our previous contract gave us an opportunity to be in the same cage as high-end custom hardware that can protect the game far better than in the past. Since we cutover in November, we have been hit with some of the biggest attacks I have seen since we took over. One attack was 200Gbps. That is 10-20x what were seeing in the past. Our new provider blocked nearly all of the attacks with zero impact to players, and the few that got through were analyzed immediately so our rules could be patched in minutes. I will continue to push for status on the EU servers, and I’ll post here when I have more details. Thanks, Matt

Hi all, I’ve seen some good responses to the OP’s issue. I appreciate the players who took the time to share their view point. I’m going to lock this thread, as it seems to have reached its conclusion. Thanks, Matt

Hi all, I was hoping for more of an update this week, but the holiday got in the way. As of Friday, we believe we have fixed the massive lighting issue that was plaguing us during December. I’m waiting for a build to evaluate performance. For console players, we noticed that both Sony and Microsoft have required upgrades to their latest SDKs before we can resubmit again. The work on the Sony SDK is complete, and we are 50% done with Microsoft. Looking forward to seeing progress next week now that vacations are done. Thanks, Matt

Hi all, This is a quick note to let you all know that Lixil has had an unexpected life situation that requires a change in her role. She is staying with the company but stepping down as the Community Manager. We are actively looking for a replacement. In the meantime Ritual and the rest of the team will be filling in. This is a bit of a scramble for us, but I hope to introduce the new Community Manager soon. EDIT: Unexpected. Thanks, Matt

Hi all, Missed the holiday week while we had reduced staff answering support tickets. Definitely saw a rise in tickets during that time. As of today we have: 497 unanswered tickets (up 159 over the holiday) 677 total tickets (up 137 over the holiday) We are answering tickets from 12/13 putting us back to 21 day turnaround. We have a new CS starting on Monday, and all staff will be back in the office to help bring this back down. Thanks, Matt

Hi all, All of our networking partners are back from the break. It was extremely frustrating to see everyone leave for the holidays with our servers not back online. I'm told my team will have access tomorrow (Friday) and that we can start installing images over the weekend. If everything holds true, then EU servers will come online during next week's patch maintenance. I will be extremely happy to have this put behind us, so we can concentrate on the next phase of world consolidation and local districts. Apologies, Matt

We are working on this issue.

Hi there, We are looking at that issue. There were other more important things to fix first. Thanks, Matt

Hi there, We appreciate you bringing it to our attention. Thanks, Matt

Hi all, Unfortunately we are still working through issues with the EU servers. All of the boxes arrived as expected, however we hit a very frustrating snag - it appears the configuration we laid out was not explicitly followed, and the game server software wouldn’t run on the CPUs due to missing instruction sets that we require. The team has expedited a new batch of servers arriving tomorrow. I’ll keep you posted. Thanks, Matt

Hi all, I missed the update yesterday, and I have been out all week with a nasty cold, so I don’t have a lot of new info. We did release a build to SPCT last week (our first in a while due to the network migration). That build had some problems and ran horribly bad. Looks like it was missing a huge chunk of the work that was completed. The team is organizing a new build for testing, and I’ll post results as soon as I have them. Hopefully all the APB players are gearing up for a nice holiday break. Thanks, Matt

Hi all, As of today we have: 338 unanswered tickets (-62 from last week) 540 total tickets (-65 from last week) We are answering tickets from 12/4 bringing us to 16 day turnaround on new tickets. Thanks, Matt

Hi everyone, We've just put up details for the Holiday Event 2019 including lots of new rewards. You can read them here: https://www.gamersfirst.com/apb/news/2019/12/18/holiday-event-2019 For PC, we're doing a new event called "Holiday Cheer" in Beacon. For Console, we still can't patch, so we'll be running the Elves and 12 Deaths. Holiday Cheer is coming to Beacon! A new drug has hit the streets of San Paro, will you be able to resist the scourge that is… “Holiday Cheer?” Those brave enough to try and resist the hallucinogenic properties of this powerful drug and fight to keep the kids safe from all that is merry can head to Beacon straight away! Be on the lookout for the Prince of Pushers… The Nutcracker Prince, with his “snow ball” launcher full of “Holiday Cheer.” Will you become dosed and join his army of tweaking Nutcrackers, or fight valiantly to keep this menace off the streets? Either way you’ll be earning new rewards! NOTE: We hit a snag yesterday with some incompatible CPUs that were accidentally ordered by our vendor and shipped to the new EU datacenter. Replacements have been organized and shipped locally. I'm working with our team over there to expedite things. At this point we're delaying the Holiday Event till Monday the 23rd in an effort to try and get EU servers online before we start. Thanks, Matt

Hi there, We are definitely running late on this event, but I'm the one holding it back till we can get EU servers online. In my opinion, it makes no sense to waste all the hard work and effort that we put in this year, when a significant amount of the population wont play due to terrible latency. I'm still waiting on an update from our EU hosting vendor to make the final plan. Thanks, Matt

Hi all, Our network provider has informed us that the EU hardware has been delivered to the new datacenter, racked, and they are working on installing OSes. I believe the plan is to give us access tonight, so we can install the game server build. I’m sharing what I’m being told, but honestly everything involving the new hardware has been 3-4 days later than they tell us. For those that have been asking, the hardware was ordered and shipped by our network provider that we lease the boxes from. Unfortunately, we had no input or window into any of the customs paperwork or issues, so I can’t shed any light on what happened there. Thanks, Matt

Hi everyone, I'm a little late getting this posted today, but here is the update. It's easier to start with what we fixed: - Baking Lighting Not sure what happened the first time around, but Financial ran in 3 hours over last weekend once we restarted it. All major districts are done now. - Memory issue This is mostly fixed. We're operating within proper limits on all platforms now which unblocks console development. There will likely be some ongoing work to reduce the memory footprint. - Performance Once proper lighting was baked, we did uncover an underlying problem with the way we re-wrote interactive features. Many of them include lights, and previously those lights were destroyed and rebuilt every time a player would go in and out of range of the feature. That had the upside of making sure the game was only processing and rendering lights within a specific distance, but the horrible downside of invalidating all the baked lighting forcing those lights to run as dynamic lights. Consoles got away with this, because they are locked at 1pm, so that version never runs at night. We had already fixed the code so that baked lighting is used all the time, but then we realized that the game was processing and rendering every light attached to a feature... every frame... regardless of how far away it was. This new issue got fixed by Friday, and the subsequent performance tests were very very promising. What is left: - Still missing our APB 2.1 Test World environment We can't deploy anything for SPCT to test yet. - Pedestrians popping through raised floors We didn't have quite enough time last week to look at this. - Console specific updates I have previous listed a number of console SDK issues that need to be fixed so we can submit to Sony and Microsoft again. - Lots of minor issues through out the levels These are gameplay bugs reported by SPCT and our internal QA such as bad collisions on specific rooftops. As I have previous stated, I wont hold up the release to OTW for these kinds of issues. - New localization Lots and lots of text in the game is new or changed, and it needs to be translated to French, Italian, German, Spanish and Russian. Again, I wont hold up the release to OTW waiting on localization. I may not even hold up the full public release. We'll fill this in as soon as we can, but I want the game to settle down first, so we can make one big pass. Thanks, Matt