MattScott

Hi everyone, It's that time again - we have reached another two weeks since my last update. We are still at the ‘get the editor running’ stage of the build process. The script generation commandlet needed some unexpected additional work. We identified several new issues while building the scripts mostly related to serialization of classes. SIDEBAR "What is Serialization?" I don't like using jargon that isn't easily understood by everyone, so since this update is pretty short, I'll explain what "Serialization" is. At a high level serialization is the concept of translating structured memory into a format that allow us to store it - more specifically APB data is stored in files on disk. Structured memory is another way of saying data structures or in-game objects. These are things like 3D mesh data, Gun stats, or data that controls how parts of a district show up in a mission. The critical (and exhausting) part of serialization is making sure that each member of a data structure is translated at the proper size into a format that can be reversed back into memory giving us the same original value. There are a lot of different reasons why sizes can be problematic or truncate data such as 32-bit -> 64-bit conversions that might have smaller or larger definitions of a memory block. This is made worse by the fact that many parts of APB had hard coded block sizes set at the top of the data structure. So we have to go in and find every one of them to make sure they are all handled the same. END SIDEBAR And of course... there are lots and lots of data structures for APB (and Unreal). To address some of the issues we found, we spent some additional time pulling across relevant code sections from the 2.x upgrade back to 1.30. Our plan has not changed, but this process of pulling across code is still going so that serialization works properly. Then we can rebuild and regenerate APB: Reloaded’s Unreal scripts.. and then the editor will load. Our current benchmark is still to get APB’s Social District map loading. Thanks, Matt

Hi everyone, It’s been another two weeks since my last update. Here’s another update on our schedule. We did pretty well. At this stage, every project compiles and links under 64-bit without warnings or errors! This is a significant milestone, as it means most of our time will no longer be spent merging in code, but instead debugging and getting the editor, game, and servers running. We are now at the ‘get the editor running’ stage of the build process. This involves resolving startup / initialization issues with dynamic link libraries, initialization, and instantiation of core engine components. Then, we will ensure that we can rebuild and regenerate APB: Reloaded’s Unreal scripts. After that, we will deal with existing disk-serialized package misalignments that would prevent maps from loading. So right now, the focus is on instantiation of core engine components. When we have all the items above sorted, then we will be able to attempt to load the Social District map. Thanks, Matt

Hi everyone, It’s been exactly two weeks since our last update. Our progress continues with APB 1.30, and I promised to give more frequent updates on a bi-monthly schedule. My hope is that keeping that schedule will provide some insight into how the team is coming along. Firstly, we have cleaned up all of APB’s dependency projects to compile under 64-bit. For the most part, we tracked down the appropriate libraries that would work with 1.20’s codebase out of the box in 64-bit, and in some cases, we modified existing libraries ourselves to fit the older version of Unreal. Between middleware integrations (such as Scaleform and Wwise), a rewrite of some 32-bit specific optimizations and texture utility code written in assembly, APB’s custom Music Studio and Web Browser libraries, Windows API support, and APB’s User Interface and Editor modifications, there has been plenty of work completed. A total of about 3,000 warnings and errors have been resolved to date. Next, there is only one more project to finish converting to 64-bit – the gameplay code for APB itself. After the entire codebase compiles without warning or error, we will have to resolve issues that may arise from the process of linking the projects together. And then we can fire up the Unreal Editor and load a district. From there, we expect to find problems with serialized package structure alignment similar to what we encountered with the 2.x upgrade. Since most of these issues were already resolved in 2.2, I'm hoping we can reuse that work to do an ‘in-place’ upgrade of APB 1.20’s serialized content packages to support 64-bit architecture. Even though we missed our previous mark of wanting to be in editor attempting to load packages by the next two-week update, we still completed a significant amount of work in a small amount of time, and we have not encountered any unknowns that we haven’t overcome before. I’m hoping that we will be in-editor resolving issues with APB’s Social District map for the next two week update. Thanks, Matt

Hi everyone, I've locked this thread similar to our original Engine Upgrade thread, so that I can post on a regular basis for people to follow. This week I'm traveling to Europe in prep for Gamescom next week. Then I'm staying over another week to wrap up business there before I return in early September. With that in mind, I'm setting a goal of posting every 2 weeks. Right now, we’re progressing nicely through with APB 1.30 (also known as the 64-bit port of APB). We’ve been monitoring feedback after my blogpost came out, and I get it. There is a lot to be frustrated with ‘yet another engine upgrade’ and the lack of tangible progress for players. Here is my update this week on exactly what actual work has been completed: 1) We have finished porting most of the core engine functionality, excluding APB-specific gameplay code, over to 64-bit. This involved a lot of different changes, most of which are backports from the 2.x code, however, some are entirely new changes to the codebase so that 1.20’s functionality is fundamentally preserved. The engine code had thousands of errors and warnings that we had to work through. At this point, I'm assuming that any warnings are really errors that need to be resolved before we ship 1.30. Here are specifics for some of what we had to do: Set up projects to have a consistent struct member alignment across the entire project (backport from 2.x) Ensure compatibility with the VS2017 toolchain. Identify and set up versions of third-party libraries, such as Scaleform and PhysX, that will suit a 64-bit client build of the 1.20 codebase Changing hard-coded, casted pointer types from implied 4-byte pointers to implied 8-byte pointers. Clean up declarations of local variables hiding class variables. Clean up declarations of local variables hiding other declared local variables. Modify the codebase to support newer versions of specific middleware libraries and ensure functionality is not compromised 2) We’re in the process of cleaning up the ‘core’ components of APB, such as the middleware made specific for APB. This is mostly complete but may need further adjustments. 3) We are beginning to do the same conversion process for the Unreal Editor components. Our next steps will be to finish converting the Unreal Editor components, and the APB-specific components. Once that process is done, we will attempt to launch the Unreal Engine Editor and attempt to get a map loaded in-editor. This was the same strategy we successfully implemented for the APB 2.x upgrade before we started beta testing and realized that the underlying implementation in 2.x was never going to perform very well. This time, we have taken the right steps to ensure (crosses fingers) that the Editor process will go smoothly. My hope is that there will be minimal time from getting a map loading in-editor to having it running in our actual client/server setup. That will allow us to do some performance and stability testing on both the APB client and its district servers. I'll be looking for opportunities to share ‘client havoc’ as it comes up, as that seems to be a community favorite. It's been a little over a week and a half since my last update. I'm hoping to do my next update in 2 more weeks once we are in the editor resolving issues with APB’s Social District’s map. Thanks, Matt

Hi all, I'm going to ask that you reserve judgment on STFU. We have a slew of information and articles coming out that will explain our "Web 2.5" hybrid approach. I'm tempted to remove the word "NFT" from anything connected with STFU, because they have such a bad reputation. STFU is a big experiment, but we're well aware of the market. It's would be folly to try and do what everyone else is doing. We have no interest in creating meaningless assets to make a quick buck. I am very interested in establishing clear ownership of content created by players so they can earn real revenue from it - if they want it. Stay tuned. Thanks, Matt

Hi everyone, I know it's been a while. I didn't get sick or die. I've just been buried with the studio and getting some work-for-hire projects shipped. Little Orbit is very close to announcing a couple new things we've been working on, but I'm way overdue for an update on APB. I finally unpinned the APB 2.0 thread, and I'm starting this one. Here is my first APB update in a long time to explain where we're headed. https://www.gamersfirst.com/apb/news/2022/8/5/message-from-matt Thanks, Matt

Glad you guys had fun!

Hey everyone, Sorry about the outage. We are back online. Thanks, Matt

Hi everyone, I hope you enjoyed the Thanksgiving holiday. We had some days off, but still made decent progress. At this point everything in Social is working except for collisions on the skinned meshes like statues. We started work on Financial this week as well. There are some issues with streamed assets that are getting loaded twice due to older and newer code running at the same time. We'll dive into that next week. For now, here's another video with progress. Working collisions are: Joker Ammo vending machines Mailboxes (specifically, the spot in Social where people gather most often) Marketplace Kiosks Static Meshes Streamed Terrain 2021-12-03 15-42-55.mp4 Thanks, Matt

Hi everyone, Apologies to the community. It's now been 45 days since my last update. I have been in Europe for the last 5 weeks, and my schedule was been incredibly busy during the trip. If I had to choose a single word to recap progress, it would be: "frustrating" Back on October 2nd, we had a big breakthrough, and I was feeling really good about next steps. You can see from the screenshots in that post that we had finally gotten collisions working on the sidewalk. That update was posted minutes after we got a build that worked. What we didn't realize at the time, was that if you stepped onto the grass, you immediately fell through the world again. Additionally, several different types of placed props and static meshes in the map also had collision problems. During the weeks that followed, we fell down a lot of rabbit holes attempting to find the issues. While we could have individually made those objects conform to the new collision system, doing so would have been wildly unsustainable. Once we start work on Financial, there are thousands of objects and checking everyone of the them would be a nightmare. Instead, we chose to take additional time to step through line by line in the collision code on the current live version of the game versus the engine upgrade. We need the conversion process to properly rebuild each map and all of the collisions in one pass. After a bit of digging, we were able to identify the core issue for terrain, so that is working properly now. As with all things APB, of course there are still issues. For some reason we have some oddball props and items that still aren't colliding properly. So the team is still working on restoring old 1.20 collision code to handle every specific case. Below is a video running around Social with nearly all of the collisions working. As I stated in the last update, ignore the lighting, we haven't turned that code back on yet, so everything is either fully lit or not lit at all. You can see once we get to the placed crates, those don't work. And the statues don't have collisions. As soon as we have those sorted out, we will reprocess all of Financial to see how that district does. apb-engine-update-2021-11-19.mp4 Thanks, Matt

The game is back online!

Hi everyone, Sorry about all the maintenance and downtime. We've been adding pieces to help with performance. I was notified about the downtime that started this morning, but it slipped past me and I didn't get a chance to post it. We're currently down for a bit longer. I'll post when the game comes back online. Thanks, Matt

Apologies. This one slipped past me. There was another resource increase scheduled that I forgot to post.

We are back online!

All everyone, We have just started some maintenance to add more resources for the game. Be back soon. Thanks, Matt

As previously confirmed. Servers are back online.

Hi everyone, We've taken the servers down for maintenance to address a couple issues that popped up. I don't yet have a time estimate, but we'll try to be quick about it. Thanks, Matt

Hi everyone, After two years, we are live! Please bear with us for any speed bumps. I have no idea how the servers will handle traffic. You can download the game here: https://www.gamersfirst.com/fallenearth/ We are working on the Steam version next. Thanks, Matt

We are knee deep in launch prep, but I wanted to drop in and thank everyone for their amazing response to this news! The community response is so much more than I expected. I will do my best. We are making good progress on other projects, so I'm looking forward to having more free time to interact with the community.

Hi everyone, It's been far too long between updates, but as some on you may have noticed I made a post on Twitter on the 2 year anniversary of taking down the Fallen Earth servers. After that I posted again in response to a tweet. While I should have posted that here to begin with, I am happy to confirm that "Fallen Earth Classic" is going to be a real thing. We are now at the very late stages of getting the servers back online, and as some of you on Discord have noticed, parts of the game are responding again. There will be an announcement soon on the day of launch, but before that goes out, I want to raise a couple discussion points here to set expectations properly. 1) I'm doing this in the spirit of fun and just getting the game back online. There were decisions that had to be made in order to pull this off. As we go through the rest of my bullets, just know that in each case I had to balance getting this back online in a reasonable amount of time versus the laundry list of things I'm sure players would have wanted. We did our best. My hope is that players will enjoy the challenge of re-conquering the wastelands from scratch while they wait for the next version of the game. 2) Steam doesn't work right now https://store.steampowered.com/app/113420/Fallen_Earth_Classic/ is now live. We are waiting on Valve to restore the old game. As soon as that happens, then we'll post an update here. 3) Previously, we had some major issues hosting the game. Nothing changed here. Those issues are wide ranging. They include being forced to host on old, out of support 32-bit Linux servers, attempting to work around a horribly bloated database with item duplicates everywhere, and expensive external hosting costs. This time around, we worked through each of those issues as we built the plan for bringing the game back online. We successfully recompiled under 64-bit and we're using new operating systems. We also completely wiped the server. (SIDE NOTE: I sometimes refer to this as "servers", but only because the physical implementation of Fallen Earth requires more than 16 different services spread across a number of boxes). That means that you wont have any previous progression - including characters, items, or G1 purchases. I know that's a bummer. Trust me. We looked at other ways to do this, but many of them would have required a lot more work, testing, and time. This is an experiment. We wont really know how the game is going to perform till we get a bunch of players online. Which leads me to... 4) There will be no paid options for the game. For various reasons, I don't want to monetize this version of the game. It is truly free-to-play (refer back to #1 above). I'm going to look at turning on the highest level subscription rewards for everyone, but there wont be any ability to purchase anything from the online store. With that out of the way, I look forward to seeing you guys online. Thanks, Matt

Hi everyone, Sorry for the delayed update. I know there's been a few weeks without an update. In general, I prefer to only share significant milestones with the community. As I previously mentioned, our focus has been on making sure we can load the converted 1.20 Social District in our 2.2 build. With a bit of software engineering work, we were finally able to get the Social district to load in the editor, though without proper lighting, shadowmaps - and importantly, collision did not work on the legacy feature data meshes. APB is an extremely large and complex game. It has taken extreme discipline in addressing specific parts of the game that do not work. When we go to fix a system, such as collisions, it's very easy to stray off-course and try and fix two or three other systems, only to have two more break in their place. By hyper-focusing on feature collision, we were able to track down where the system lost its data, and also add in collisions automatically on level conversion. As as a result of that effort, we now have collisions working in the map. The team would like to share with you a few screenshots of the 1.20 Social District in our 2.2 client build. Veterans of APB will notice the lack of palm trees. This is substantial progress. We have a minor crash to address, and then the team will start work on the other maps and missing functionality. Thanks, Matt

Hi everyone, It's been nearly a month since my last post, and for that I apologize. As I have mentioned previously, these updates are sometimes tricky to write when we are deep in the weeds fixing issues. It always feels like we are right at the cusp of a big break through, so it's tempting to wait "just one more week" till we have something more to show. I don't have something big to show - yet. But I do have significant progress to talk about. I'll admit, the last 4 weeks have been a slog. We've hit very nuanced issues surrounding exactly how to load the new 1.20 data we were trying to preserve. These issues crash our conversion process. Each time, we spend nearly a day converting packages to 2.2, hit a crash, go back to the 1.20 code, step thought side by side, find the problem, work out how to fix it, and then re-run the lengthy conversion till it crashes again. Some of that nuance comes from the fact we simply can't take all of the 1.20 data packages. There are specific APB 2.x screens or console-based packages that we need to preserve and merge. It seemed by the end of last week that we had finally addressed all the bugs, and that we might get a full map to load into the editor. Then we hit actual bad data in the 1.20 files. Empty meshes and broken references to missing 3D objects. The works. It is a mystery to me how APB 1.20 runs as well as it does. But each of these new problems would break as APB 2.2 tried to load those files. So this week we have been playing whack-a-mole with bad data and adding code to handle each case. Today, the entire APB 1.20 Social map loaded successfully in the editor. No palm trees. No odd foliage. Just the live version of Social including all of the feature data we need to move forward running in the APB 2.2 editor. It was a pretty awesome day. The team pushed hard to get Social up and running as an actual district in APB 2.2, but we hit an entirely new problem related to how levels are loaded at runtime. We will be addressing that first thing next week. Our next steps are: Get Social running at runtime so we can snap off a couple screenshots and run a benchmark against APB 1.20 Run the conversion process on Financial and run a benchmark against APB 1.20 Convert the rest of the maps. Re-bake all of the lighting with UE 3.5's Lightmass. Fix the main rendering shader so that lighting is applied properly for the special case light maps that Real Time Worlds created. Assuming the performance is where we want it, then we will get a small group of community testers onto our test servers. I know this has been a long process, but no one has attempted this new direction before. It's uncharted territory. As I have stated before, the original team at Reloaded felt it was impossible to port the game this way. But today we saw our first big sign of progress. Hang in there with us. I have lots of plans, but they all rest on making the game sustainable. Then we can dig into the fun stuff. Thanks, Matt

Hi everyone, We made some good progress on the loading of older 1.20 packages this week. We are still not able to load the Social district. However, I feel like we are very very close now. As a reminder, Feature Data was baked into the 1.20 maps for displaying static mesh actors and culling/rendering them under appropriate scenarios. That data was removed when the maps were upgraded to 2.0 (console) and 2.1 (Engine upgrade PC). While we have upgraded various packages regularly since the original work in 2014, the maps themselves were upgraded once way back then, and then permanently modified with new art. No attempt has been made to upgrade them from 1.20 to 2.x since then. Here is some of the work we finished on the Feature Data this week: We're now parsing and loading the feature data correctly in most scenarios. We added support for alignment of lightmap data under 1.20. Byte alignment is a big problem we recently uncovered. The original 1.20 maps are mostly 4 byte aligned, but most of the new engine is 16 byte aligned. Of course, we cannot actually use the lightmap data stored in the maps. But generating new lightmaps should be trivial, provided we re-save the package and discard of the old lightmap data. The same goes for shadow maps. We've made progress on aligning the shadowmap data, and will treat it the same as light map data. We will remove that data and re-save when we generate lighting. Another issue we have been sorting out are Prefabs. Prefabs are references in a map to instances of a static mesh. This was used heavily in 1.20 and removed for 2.x. Unfortunately, in some cases now the saved Prefab references in older maps point to invalid engine classes. This coming week we are going to work on cleaning up all the bad references. We're also utilizing a new test map that contains feature data in 1.20 to get the data ported over: The APB Login Level. That single map, which can be tested by simply loading the game, contains examples for most of the data we're porting over. Most of it is off-screen and can't be seen by player, but the map actually contains parts of the Financial District and is small enough so that we have sample data to test without running into red herrings. Once we can successfully load this map into 2.2, we will try porting over the Social district from 1.20 again. Once that district loads properly with collisions, we will attempt to load the Financial district from 1.20. That will let us do another internal benchmark, and continue the work from there. Currently, we don't see too many blockers in getting this task done, but it is worth acknowledging that none of this has been attempted before. There's bound to be ways it will break. Once again, there are no fun pictures to show the community yet. We're still aligning data and almost all of the work thus far is code work. I'm feeling good about this week. Ready to show more as soon as possible. Thanks, Matt

Hi all, Apologies about the lack of an update last week. There wasn't much exciting to post about. But my team sees you guys in Discord, and I know you're waiting for more news, so I'm going to try and provide that now. We are still stuck migrating data. So far this week, we have focused on cleaning and stabilizing the codebase so we can take the next step of merging in data from 1.20 to 2.2. We fixed the issue related to Static Meshes loading properly, and then resolved an issue involving Octree data. In order to do this, we had to take several steps backwards. We ultimately decided to focus on getting the pipeline of feature data loading but with placeholder data, so we could make sure the resulting data works without crashing in-game. We were able to complete that work. It doesn't render yet because there is no feature data to render. But at least we know the underlying structure works. Now we are working on porting over 1.20's feature data to fill in the blanks. Here is where I made a pretty big assumption about the project overall. I assumed that over the last 10 years of engine changes, the previous devs always maintained backwards compatibility. Little Orbit constantly merged 1.20 packages up to 2.1 for the last couple years. But it appears that data between 1.20 and 2.2 aligns perfectly in some places and not in others. These are all in areas that never popped up before. Our contractor is currently working on a solution to identify the version of the data we're loading so we can use the proper structure in memory. Basically, we know how the data is supposed to be aligned, and we know 1.20 data is saved all with the same version number. That means the data should convert one-to-one to the new format, but only if we add back in the original structures from 1.20 across the board - even they aren't used anywhere in the actual game. The process involves going through, line by line, about 200 separate blocks of conditionals and adding in the proper support. We have to pay close attention while we upgrade each condition because if the wrong comparison operator is applied, it will break the entire conversion process. Once that process is completed, then we can finally extract the old data we need from the 1.20 packages and resave it in the 2.2 packages. A lot of this work is purely code based. We aren't loading maps in the editor except to run commandlets or to test how something functions in the scene graph. So no screenshots yet. Our progress right now is measured by the lack of crash dumps and warning text on converting the Social district map. I'll grab more shots once we've made a bit more progress. Thanks, Matt

Hi all, This week had some ups and downs, but overall I would say there's a net positive amount of progress going on. We had an issue where loading Static Mesh data was crashing in some cases. That seems to be fixed now, which means the new format that has the additional data from 1.20 is working. We will have to see if the data itself works in the next couple steps. From there, we have been in the quagmire of resaving all the game's content to import 1.20 3D objects and maps back into 2.2. We do that through a commandlet. These are small Unreal programs that don't bring up the editor. Right away we hit a crash once maps got fully loaded. The crash is part of the new code that was pulled over related to lights. The engine needs to do a single 'update' to the entire scene to activate the correct lights. That update is currently crashing when the level is fully loaded. APB's handling of lights is entire custom to RealTime Worlds and deviates significantly from Epic's techniques. For the initial upgrade, we tried backing out the RTW changes going with standard Epic Games code. Now we are going back to the 1.20 technique so we can compare apples to apples in 2.2. Once we've fixed the crash, we can continue on with the package fixup process. From there, we'll attempt to load maps and debug as-needed. Thanks, Matt