MattScott

Hi all, I'm way overdue for an update on the engine. We brought on a new resource to add some momentum and pick up several tasks from me. The new resource had experience with the engine, specifically this customized version used by Reloaded, but it's been quite a few years, and he's still getting caught up on APB specific changes. We will be filling out the team more as we go, but right now we're spending a bit of time to onboard. The primary issue during my last post were the collisions in Social, which we are using as a test level. After a number of red herrings, we determined some collisions are loading, but not inside buildings. This is because the old Building Feature code was migrated and is already working. On the one hand, this is great news, because the system appears to be working and we need it to remove 1,000s of objects from the scene that we were processing in 2.x. But right now this is bad because the Building Feature data is empty. It was created with a Real Time Worlds era tool and baked into the APB 1.x maps. If you recall that map data was wiped out during the migration from 1.x to 2.x years ago. Essentially, without the proper data, the game has no idea what building you are in or near, so it rejects loading any of the assets inside - including the collisions. We are current focusing exclusively on re-migrating the data from 1.20 to 2.2. That involves picking through and understanding that RTW-era system, and then building the tools to modify the data in that system, which is separate from any existing Unreal 3 systems. These are techniques we can utilize later to re-optimize the data, but the important part for now is that the data stays the same, and that the data loads correctly. You guys might remember the 'contrust-like' Enforcer trainee abomination from years ago. We're basically aligning the data to prevent 'creating' more of those, and to actually have the data appear as it was before. Once the data loads correctly, we should have proper collisions. We'll do some quick fixes to the shaders and then run benchmarks. Thanks, Matt

Hi all, Progress has been slow as I got into the weeds on a number of items. However, we are bringing on a full time specialist to help, so things should pick up pace. Thanks, Matt

Hi everyone, Rest assured the entire team is still here. I'll be more candid here. We followed the same strategy for over 2 years with mixed results. It makes no sense to keep following that same course and expect a different outcome. Instead, I came up with a new strategy that the existing team had avoided since the beginning of the port to 3.5. In fact, there is a segment of my team that feels my approach 'cant be done'. So I felt the need to take over this portion of the development and then hand the game back off. The work has involved tearing apart the base of the engine to re-implement several core systems from 1.20. That has meant having to go backward and get the data ported over again. This is why you're seeing all the rework. Thanks, Matt

Hi all, Doing a quick Tuesday morning update. I can now get into all of the action and fight club districts. There are a fair number of new bugs to work on. Collisions are the first thing I need to fix. Thanks, Matt

Hi all, It's been three weeks since my last update. I'll try to make this more comprehensive to make up for that. I'm going to start by acknowledging that I'm now leading and handling the majority of the programming for the upgrade on APB. That may already be obvious based on the detailed previous posts, but I figured it was worth publicly stating so you guys can understand some of the slowness. While APB is a priority, I only have so many hours in the day after running the company. There are a number of reasons why, but it really comes down to the fact that everything left to do is very very specific to APB. Simply knowing Unreal 3 or GPU shaders isn't enough at this point. No one else has enough knowledge across all the game's systems (art, shaders, etc). I still stand by this decision. The progress is farther than we've gotten in the last 2 years, but it has come with a cost to my own personal life and free time. The burnout is real. However, this is something I committed to, and I want to see it through so we can get back into full production. Also, I want to address the last couple weeks specifically. In general when Friday/Saturday comes around, I make an extra push to try and get to some screenshots for something more visual rather than these long, word heavy updates. Unfortunately, the last couple weeks have been spent in the weeds cleaning up bugs which prevented that. In every case, I felt tantalizingly close. So I held off on the update thinking that I could get the kind of progress I wanted before the weekend was up. But even with the extra push that never happened, so I ended up waiting another week, hoping for better results. With that out of the way, let's dig into progress. Before I jump in, I should note that Programming 101 says to try and introduce new code in discrete batches so you can test each piece and know that it works before moving on. This is especially important when bugs pop up, because you know exactly the code that was touched, and it's much easier to track things down. When you have to move code over in bulk, you end up not knowing exactly where to start looking when stuff breaks. I did my best to try and move bits over in chunks, but everything is so intertwined that it wasn't really feasible. Here are a couple of the "needle in a haystack" tasks that occurred over the last couple weeks. 1) Tracked down an issue that caused everything to render black (both in the Editor and in-game). This ultimately ended up as a bug in the primary BassPassPixelShader. But even when I fixed that issue, objects remained largely dark due to the missing baked lighting data. I have temporarily disabled the lighting calculations in the main pixel shader. Once I have finished a couple other tasks, I'll bake lighting and back out this change. For clarity, the game is doing all the lighting calculations so benchmarks will be accurate, I just short circuit the final output to make sure everything actually renders for now. 2) Tracked down an issue that caused all skeletal meshes not to render (both in the Editor and in-game). Skeletal meshes are any object that has bones we can animate. This includes not just player characters and NPCs, but many props and cars. The issue was related to some new transparency optimizations that had been badly ported over from 1.20. Effectively the game was treating all skeletal meshes as transparent, but then not rendering them when we render all the transparent objects. That is now fixed. 3) Tracked down an issue that was creating polygon soup in-game. I probably should have taken a screenshot of the login scene to show what this looked like, but I worried it would be too disheartening to you guys. Essentially "polygon soup" is when you have a serious problem with vertices and points for a mesh. It creates a bizarre scene of stretched polygons everywhere that look nothing like what you're trying to render. At the time, everything in the Editor rendered fine, so this was some in-game specific problem. It ended up being some bad code in an APB 1.20 system that batches mesh data to the GPU. It's a smaller system that I moved over to get the speed benefits, but there ended up being some DirectX 9 -> DirectX 11 bugs. Here is where that leaves us: All major systems ported over from APB 1.20 are working. All individual objects render in the Editor (albeit without proper lighting) The login scene renders fine (again with no lighting) I'm working on getting district maps loading properly in the Editor. This was working, but after fixing some code that supports the features attached to buildings, the maps now error because they are missing data. I need to go back and finish re-importing all the APB 1.20 content so I have the feature data that was lost. Once the district maps load, then I can spin up a district and do the benchmarks. This week, instead of showing off the almighty sphere, I figured I would include some behind-the-scenes shots that reveal a bit more about how the sausage is made. For a lot of the initial work a couple months ago, I was using a relatively small test level to slowly add objects and fix the bugs. This is setup so that I can actually spawn it and run around. This map only exists in APB 2.2, so it's not super useful for testing side by side with APB 1.20. After that I moved to an existing map in APB 1.20 that we use to test new gameplay. If you look closely, you can see some RIOT bombs leftover. It's not pretty, but it gets the job done. Thanks, Matt

Hey everyone, You've been more than patient, and you deserve a better update. As a company, we need to clear two projects off our plate to be able to dedicate proper resources to my plans for Fallen Earth. Having said that, I have started looking at intermediate ways I can put things online for you guys to experiment with. Please don't over hype this. I just realize this is taking longer than I planned, so I'm looking for near term solutions. Thanks, Matt

Hi all, Back to being frustrated. Spent the week deep into shaders and working out some kinks that led to mismatched inputs and outputs between Vertex and Pixel shaders. From the outside, you would think this would be trivial to solve. But APB generates shaders for every possible unique combination of lighting, rendering, fog, and other variables. It's a lot. So the effort this week was like finding a needle in a haystack. Run a map, catch the crash, figure out which object it is and what material was attached, and then examine why those shaders aren't in sync. More next week. Thanks, Matt

Hi all, I hope you had a great Mother's Day break. I missed my update yesterday. We made good progress last week. Serialization issues are fixed. Objects load again, and we're troubleshooting lighting now. We need to finish up one more piece, and then we will reimport all of the 1.20 packages. I had thought we would have screenshots this week. But nothing was as impressive as the Sphere shot from before. Thanks, Matt

Hi all, I was pushing right up to tonight to see if we could get something to show for the progress this week. But I want to stick with at least an update every week, and I'm out of time. Oh well. We're quagmired somewhere between re-compiling the last of adjusted shader code, sorting out an optimization that relied on 3 lighting coefficients so APB can store light tangent vectors in the alpha of each one, and serialization issues nested deep down in LOD data for static meshes. The work continues. Thanks, Matt

Sorry for the delayed response. 2.0 has the old version. We upgraded it once we started 2.1.

Hi all, Another slightly frustrating week, but only because we had to backtrack a bit. We have decided to go back to the APB 1.20 assets and reimport them. It was a tough call, but I feel it will eliminate any further hidden land mines. As we prepped to reimport, we found issues in the previous import related to how collisions were handled and a couple other items. Last week was spent going over that code. Reimporting will likely also involve a little bit of debugging and possible bug fixing to make sure the missing data can be loaded properly from the old 1.20 files and saved into the new 2.2 files. Then we can go back to debugging and fixing the issues that were blocking us in 2.2. Welcome to Development 301, how the sausage is made. Thanks, Matt

Hi all, I know many of you have been asking about Vivox voice chat. It has been down for a while, which has been a concern for our console players in particular. We have been reaching out to Vivox to try and resolve the issue, but the long and short of it is that the code in the APB client that interfaces with Vivox is quite old. Vivox has been doing us a favor by supporting us with specialized servers for some time now, but they are unable to continue doing so moving forward. We are looking at solutions, but unfortunately this means that voice chat will be down until we are able to update the client with the Engine Upgrade. You can follow my regular updates on the subject here. In the meantime, I encourage those who wish to chat with their APB friends to make use of 3rd party solutions until we can get the in-game chat running once more. Thank you for your patience and understanding, Matt

Hi all, This was a bit of a frustrating week. The main issue right now is a combination of data that was discarded in the original upgrade and the fact that between Unreal 3 and Unreal 3.5, Epic started compressing lighting textures. Epic did this to reduce texture memory by 1/3rd for Directional Lightmaps from 3 to 2. So all of the maps for APB 2.1 ran code during the conversion to get them into the proper format. However during this conversion, the original dev team also dropped other data that they felt they wouldn't be needing. In order to finish, we need that missing data. On top of that, the newer districts have been modified so they are no longer 1:1 with the APB 1.20 districts. This puts us in a weird spot. We can try keeping the newer APB 2.1 districts and then work out how to recreate the deleted data. Or we can modify the conversion process to properly handle the data they deleted and then go back and re-convert the APB 1.20 districts to APB 2.1. Doing that will mean losing all the work that Reloaded Games did on the newer districts to spruce them up. We'll make the decision on that this week. Either way, we are adding new resources to the upgrade project next week that will be able to focus on specific problematic areas, so I'm hoping that progress will pick back up again. Thanks, Matt

Hi all, I took off last weekend to celebrate Easter. I hope everyone got a nice break. There isn't much to talk about this week. We can now load very complex scenes into the Editor and they render fine. There are bugs with Decals and Skeletal Meshes that are being hunted down. We tried running the game to see how the login scene rendered, and it wasn't pretty. There were a number of really bad shader issues that made that scene look like a nightmare. We'll be working on clearing the Decal and Skeletal Mesh issues this week so that all levels load properly in the Editor. Then we'll start troubleshooting in-game rendering which goes through a much more complicated pipeline. Thanks, Matt

Hi all, I'm going to try and keep up the schedule of posting at least one update a week (even when they are small). We are now on Step 2 from roadmap I posted here. This week was spent testing, fixing, and testing again the work that has been to done. So far we have merged a ton of code, recompiled UE scripts, recompiled shaders, and patched up that code for the two systems we are merging over to get the editor fully compiling and loading without crashes. As I mentioned in the roadmap, now the goal is to get load a simple scene. For this first test, we created a small scene with a sphere primitive in APB 1.20 that has the default material on it. After fixing a number of crashes, we finally got that map loading successfully in APB 2.2 (the internal version for this new expanded version of APB 2.1). Unfortunately the sphere itself didn't render in any of the editor panels. So there was still work to be done, but we can see that all the data is there properly. Just to experiment, we spent some time attempting to load a more complex scene. For that test we used the simplest APB map that can actually be loaded by the game as a district to play in. We use this empty district to test various gameplay elements while they are being worked on. Again, this is a map that was created in APB 1.20 that we tried loading in APB 2.2. The reason we're loading APB 1.20 maps is because that engine fully supports the two systems that are being merged over. So those maps have data that we can use to properly test. And after a bit of hunting down crashes and bugs, we got that scene to load. Right now there are about 10 objects in that scene that are dismissed by the editor due to bad data. But in all cases, they are objects from the two systems we're merging over. Aside from a couple brushes, nothing rendered properly in that scene either. From there, we set about diving into the scripts, shaders, and code to find the needle in the haystack preventing objects from rendering. For the coders following along at home, we use a Visual Studio tool called the Graphic Analyzer for this. That tool allows us to capture a renderer frame and then step through every instruction sent to the Graphics Processing Unit to see what the data was that it sent. To make this exercise a bit easier, we took a rendered frame from APB 2.1 using the same scene that renders properly and compared them side by side. The tool itself is invaluable, but it has one big drawback - it crashes very very frequently. Fortunately the frames are saved, so it's fairly easy to reload. But that involves finding where you were and continuing another half dozen instructions before it decides to crash again. Along the way we discovered lots of places where code hadn't quite been merged over properly and where some smaller systems were overlooked entirely. Friday was pretty frustrating, because we knew were close but couldn't quite get anything rendering before the close of business. So today, we put in a little extra time, and finally got the test sphere rendering properly. Tada! Impressive, right? Behold a completely unlit sphere. Look upon its greatness and marvel. I know this does not seem impressive - at all - but given how much of the code in this game we tore up, it's good progress. There are still bugs with rendering. Some panels in the Editor go completely blank at odd times and you have to reset their view to get them start rendering again, but this demonstrates a clean path through the most basic rendering pipeline. It also means once we clean up the leftover issues, we can get into the heart of Step 2 - adding the rest of the code one chunk and a time to support the new rendering pipeline for the 2 missing systems and baking out the missing data in APB 2.2. As an final experiment this week, we also loaded our simple gameplay scene and that rendered as well, but I can't post a screenshot because it contains some proprietary info onscreen I can't share. Till next week. Thanks, Matt