MattScott

Hi there, I can appreciate this comment, and I agree that we need to get the Engine Upgrade out. However, that’s not as simple as you think. I make the effort to post on a weekly basis to talk about our progress. You can read it here: Thanks, Matt

HI all, I missed yesterday's update, but the TL;DR is that this week was disappointing. In an effort to explain, I'm going to talk a little bit about the innerworkings of the APB engine. I'll start by sharing performance from scenes that have no actual gameplay in them. These are small contained scenes with not a lot of objects in them. For the Login scene on my test machine, APB 1.20 (Live) runs at an average of 194FPS and APB 2.1 (Upgrade) runs at an average of 221FPS. We tested a number of other non-gameplay scenes, and clearly our new multithreaded renderer is faster - even with the more complex lighting and more objects overall. However, that is not true in Mission districts. These are much more expansive scenes where it's not feasible to render everything. As an example, in Financial there are 30,000+ objects. There are also a lot of special cases where task objects can be enabled/disabled, buildings can show various levels of detail, NPCs animate and move, and lighting is combination of Time of Day + lightmaps. All of that requires special case code. As mentioned previously, the Engine itself has two main threads - one for game play and one for rendering. The rendering thread performs all the CPU functions necessary to throw geometry to the graphics card (GPU), which then splits those draw calls into more processes. The trick to getting the game to run fast is to balance the tasks in each of these three areas so that from the beginning of a frame to the end, they take the same time to run. To help optimize and balance performance, we have a number of tools that do performance profiling and tell us general areas, shaders, or functions that are taking a lot of time. This part can get very tricky because we have various versions of builds we can make such as Debug, Release, and Final builds. Debug is the slowest, but has the most diagnostics information we can see. Final is the build we ship, but it provides the least diagnostic information. Also running the game at a slower speed can wreak havoc with the balance of threads, which massively skews profiling results. With all of that in mind, a couple weeks ago we zeroed in on a performance bottleneck in the render thread where we do Culling (and its associated functions). This area of the code got reworked when we implemented multithreaded rendering, and clearly it is now under performing. Culling is the process of eliminating as many items as possible from being sent to the graphics card for rendering. Even though GPUs are very fast, it still takes time to copy data between the CPU and GPU, and we take a hit every time we switch materials/shaders between rendering each object. Since APB is so old, our version of Unreal has no "batching" or "instancing", which is what allows modern engines to push so many objects. In Financial, each frame we end up culling those 30,000+ objects down to 800-1000 objects that get processed for rendering to the GPU. So Culling is very important, and we use a bunch of different techniques to cull including render distance, object state, and clipping against the view frustum. All of that starts with accessing the scene objects in an Octree, a 3 dimensional grid that sub divides each cell until only a few objects are present. Each frame we start by looping down the Octree to eliminate objects that are behind or to the side of the player and out of view. The Octree is both our best way to cull, but also it's the hardest thing to multithread. This last week, we took a stab at multithreading that. This was a massive effort, and it ultimately didn't work. We got the code working, but on Thursday night when we started testing, we noticed that some objects in the scene didn't render. By Friday night, we figured out that the original devs on APB store scene data for objects in two places, the Octree and a main list of objects in the scene. For various reasons, those objects don't match. So we need to rework the approach and try again this week. Thanks, Matt

Hi all, Lots of support requests this week. While there are a couple straggler Keys to the City tickets asking for back grants, we stayed on top of things pretty well. As of today we have: 188 new tickets (which is lower than last week) 328 total tickets (which is lower than last week) We are now responding to tickets from 1/31 (except for a couple KTTC tickets) which puts us at 15 days to respond. Thanks, Matt

Hi all, Not much to report this week. Work was finished a number of bugs, but the focus has been on performance. Fortunately it looks like the team may have found the largest performance bottleneck on Friday. Looking forward to sharing the results. Thanks, Matt

Hi all, We had a bit of a rough week that required us to split time in Customer Service between older tickets and immediate item delivery issues on GamersFirst.com caused by moving the Citadel database. Everything has smoothed out now, but we fell behind a little overall. As of today we have: 195 unanswered/new tickets 336 total tickets We are now responding to tickets from 1/25 (almost 1/26) which puts us at 13 days to respond. Thanks, Matt

Yes. We are prioritizing those above others, but there really isn't a need to open a ticket now. We're working proactively through about 100 Citadel accounts that were affected. Some of these were accounts that missed the delivery of an item going back as far as November. We got about half of them sorted out today, and we'll hit the rest over the next 2 days.

Hi there, You should have received an email receipt for the purchase. You can just paste the details of the receipt to Support. Otherwise, you can let Support know what you purchased, and they can look up your account and find missing items, but that might take a bit longer. Thanks, Matt

Hi everyone, We had an issue with G1 purchases this morning when Citadel came back online. For a period of about 3 hours, item purchases weren't being delivered to players on that world. This was due to an issue caused by moving around the database during maintenance. The issue has been solved, and all purchase should work now. However, for anyone who is still missing weapons, you'll need to open a ticket at http://support.gamersfirst.com. Give them the transaction id from your purchase, and we'll escalate delivery as fast as possible. Thanks, Matt

You may want to experiment with different action districts. Some are West Coast, and some are East Coast. The team is nearly done putting in a region ping/latency indicator. That will show up on the Advanced tab to show which servers are better for you.

The team is nearly done with this. Hoping it will go into the next major patch.

Hi all, Can you test again? We made some adjustments today. Thanks, Matt

Hi there, This could be a case of selecting a New York server or a Los Angeles server. We have action districts in both. The team is nearly done with a new patch that will show players a ping/latency indicator to the closest servers. If you get in and out of districts, you can see your latency go up and down depending on what is closer. Thanks, Matt

Hi there, We've fixed the problem, so new purchases will work correctly. However, you'll need to open a ticket at http://support.gamersfirst.com to get the items delivered. Give them the transaction id from your purchase, and we'll escalate delivery as fast as possible. Thanks, Matt

Hi all, Servers are back online with the Valentine's Day patch. This maintenance was extra long because we were moving databases and more resources into the Europe data center. This also means we finally have enough resources to turn back on threat for both worlds. Thanks, Matt

Hi all, We shipped a couple APB 2.1 builds to SPCT this week. All of them were focused on performance. The engineers were able to fix the issue that was identified last week, and the game is running a lot better. We also fixed several of the top game play bugs including an issue with destructible props and shoot windows. The biggest blockers are the memory issue on Xbox and a frame rate drop when looking around. The team is currently memory mapping the game profiling builds to identify the fixes. Thanks, Matt