MattScott

Hi everyone, It’s a very spooky Tuesday in San Paro, and with the event comes another update on the progress on 1.30! We’re still working on crashes since the last update. We’re in the process of narrowing those crashes down and have a few leads on the root cause. Unfortunately, this crash is about exciting to share with you guys as it is to fix, and it's also a blocking crash. The rest of the 64-bit upgrade is dependent on resolving this crash. For the moment, I have our most senior engineer working through the issue. The nature of the crash itself means that we need to complete the fix before the rest of the project works. Allocating multiple team members to an engineering task like this does not speed up the work in question. Other APB team members are working on creating new rewards and enabling the Halloween event, which is scheduled to go live later this week, while the team allocated to working on 1.30 continues their work. We’re still progressing. Once we get over this irritating hurdle, we will complete the rest of the tasks necessary to get the Editor loading. Still hoping that once we have the Editor loading, the rest of the components should ‘just work’, due to the nature in which we have been careful while upgrading the engine. NOTE: I've received a community letter from a group of the players that I'm working on addressing in a much larger post. We're working through the strategy of possibly splitting each section into separate forum threads that the community can weigh in on, and I'll do a live AMA as part of my response. Stay tuned. Thanks, Matt

Hi everyone, Tuesday is here, and it's time for another update. We’ve completed the work to get the editor’s script generation commandlet working. We’re in the process of making the UnrealScript ‘source’ files match the headers we’ve manually modified in various places of the engine so that 64-bit support will work. This process is honestly more complicated than it needs to be, but that's just the way it is working with this version of Unreal. Essentially, it's a chicken or egg problem. Unreal compiles scripts to generate mores source code. But the old scripts wont compile with the new version of our 64-bit source because they are incompatible. So we have to create all source code that would normally be generated to get the source to compile. Then we can process all the scripts which will replace our manually generated code with the correct version. It's a complicated process filled with potential ‘rabbit holes’, but at least it's mostly straightforward and mostly grunt engineering work. We’ve also completed the work to regenerate the “Core” project’s scripts. As of Friday, we ran into an issue parsing the source version of the script themselves that we’re investigating that currently results in a crash. Unfortunately, due to the age of the engine and the way older Unreal outputs text to logfiles, this has been a particularly painful issue as logging doesn't finish dumping once a crash occurs. So there's no help there. We’re currently in the process of identifying that crash, using a combination of three elements – working copies of the codebase that function as intended, debugger output messages, and our gut intuition on what the problem may be (insert all the community jokes here about how bad our gut intuition is). Once this issue is resolved, we will likely have to fix the rest of the project’s UnrealScript package compilation. As I mentioned above, these UnrealScript files will be processed to generate native c++ source code that needs to match the files we’ve manually merged into the 1.30 codebase. It's a crucial step so we can continue forward and launch the editor. We’re hoping to be done with the UnrealScript portion of 1.30 by the end of the month. After that, we can begin work on loading existing content packages. Our current benchmark is still to get APB’s Social District map loading. Thanks, Matt

Hi everyone, It's that time again - we have reached another two weeks since my last update. We are still at the ‘get the editor running’ stage of the build process. The script generation commandlet needed some unexpected additional work. We identified several new issues while building the scripts mostly related to serialization of classes. SIDEBAR "What is Serialization?" I don't like using jargon that isn't easily understood by everyone, so since this update is pretty short, I'll explain what "Serialization" is. At a high level serialization is the concept of translating structured memory into a format that allow us to store it - more specifically APB data is stored in files on disk. Structured memory is another way of saying data structures or in-game objects. These are things like 3D mesh data, Gun stats, or data that controls how parts of a district show up in a mission. The critical (and exhausting) part of serialization is making sure that each member of a data structure is translated at the proper size into a format that can be reversed back into memory giving us the same original value. There are a lot of different reasons why sizes can be problematic or truncate data such as 32-bit -> 64-bit conversions that might have smaller or larger definitions of a memory block. This is made worse by the fact that many parts of APB had hard coded block sizes set at the top of the data structure. So we have to go in and find every one of them to make sure they are all handled the same. END SIDEBAR And of course... there are lots and lots of data structures for APB (and Unreal). To address some of the issues we found, we spent some additional time pulling across relevant code sections from the 2.x upgrade back to 1.30. Our plan has not changed, but this process of pulling across code is still going so that serialization works properly. Then we can rebuild and regenerate APB: Reloaded’s Unreal scripts.. and then the editor will load. Our current benchmark is still to get APB’s Social District map loading. Thanks, Matt

Hi everyone, It’s been another two weeks since my last update. Here’s another update on our schedule. We did pretty well. At this stage, every project compiles and links under 64-bit without warnings or errors! This is a significant milestone, as it means most of our time will no longer be spent merging in code, but instead debugging and getting the editor, game, and servers running. We are now at the ‘get the editor running’ stage of the build process. This involves resolving startup / initialization issues with dynamic link libraries, initialization, and instantiation of core engine components. Then, we will ensure that we can rebuild and regenerate APB: Reloaded’s Unreal scripts. After that, we will deal with existing disk-serialized package misalignments that would prevent maps from loading. So right now, the focus is on instantiation of core engine components. When we have all the items above sorted, then we will be able to attempt to load the Social District map. Thanks, Matt

Hi everyone, It’s been exactly two weeks since our last update. Our progress continues with APB 1.30, and I promised to give more frequent updates on a bi-monthly schedule. My hope is that keeping that schedule will provide some insight into how the team is coming along. Firstly, we have cleaned up all of APB’s dependency projects to compile under 64-bit. For the most part, we tracked down the appropriate libraries that would work with 1.20’s codebase out of the box in 64-bit, and in some cases, we modified existing libraries ourselves to fit the older version of Unreal. Between middleware integrations (such as Scaleform and Wwise), a rewrite of some 32-bit specific optimizations and texture utility code written in assembly, APB’s custom Music Studio and Web Browser libraries, Windows API support, and APB’s User Interface and Editor modifications, there has been plenty of work completed. A total of about 3,000 warnings and errors have been resolved to date. Next, there is only one more project to finish converting to 64-bit – the gameplay code for APB itself. After the entire codebase compiles without warning or error, we will have to resolve issues that may arise from the process of linking the projects together. And then we can fire up the Unreal Editor and load a district. From there, we expect to find problems with serialized package structure alignment similar to what we encountered with the 2.x upgrade. Since most of these issues were already resolved in 2.2, I'm hoping we can reuse that work to do an ‘in-place’ upgrade of APB 1.20’s serialized content packages to support 64-bit architecture. Even though we missed our previous mark of wanting to be in editor attempting to load packages by the next two-week update, we still completed a significant amount of work in a small amount of time, and we have not encountered any unknowns that we haven’t overcome before. I’m hoping that we will be in-editor resolving issues with APB’s Social District map for the next two week update. Thanks, Matt

Hi everyone, I've locked this thread similar to our original Engine Upgrade thread, so that I can post on a regular basis for people to follow. This week I'm traveling to Europe in prep for Gamescom next week. Then I'm staying over another week to wrap up business there before I return in early September. With that in mind, I'm setting a goal of posting every 2 weeks. Right now, we’re progressing nicely through with APB 1.30 (also known as the 64-bit port of APB). We’ve been monitoring feedback after my blogpost came out, and I get it. There is a lot to be frustrated with ‘yet another engine upgrade’ and the lack of tangible progress for players. Here is my update this week on exactly what actual work has been completed: 1) We have finished porting most of the core engine functionality, excluding APB-specific gameplay code, over to 64-bit. This involved a lot of different changes, most of which are backports from the 2.x code, however, some are entirely new changes to the codebase so that 1.20’s functionality is fundamentally preserved. The engine code had thousands of errors and warnings that we had to work through. At this point, I'm assuming that any warnings are really errors that need to be resolved before we ship 1.30. Here are specifics for some of what we had to do: Set up projects to have a consistent struct member alignment across the entire project (backport from 2.x) Ensure compatibility with the VS2017 toolchain. Identify and set up versions of third-party libraries, such as Scaleform and PhysX, that will suit a 64-bit client build of the 1.20 codebase Changing hard-coded, casted pointer types from implied 4-byte pointers to implied 8-byte pointers. Clean up declarations of local variables hiding class variables. Clean up declarations of local variables hiding other declared local variables. Modify the codebase to support newer versions of specific middleware libraries and ensure functionality is not compromised 2) We’re in the process of cleaning up the ‘core’ components of APB, such as the middleware made specific for APB. This is mostly complete but may need further adjustments. 3) We are beginning to do the same conversion process for the Unreal Editor components. Our next steps will be to finish converting the Unreal Editor components, and the APB-specific components. Once that process is done, we will attempt to launch the Unreal Engine Editor and attempt to get a map loaded in-editor. This was the same strategy we successfully implemented for the APB 2.x upgrade before we started beta testing and realized that the underlying implementation in 2.x was never going to perform very well. This time, we have taken the right steps to ensure (crosses fingers) that the Editor process will go smoothly. My hope is that there will be minimal time from getting a map loading in-editor to having it running in our actual client/server setup. That will allow us to do some performance and stability testing on both the APB client and its district servers. I'll be looking for opportunities to share ‘client havoc’ as it comes up, as that seems to be a community favorite. It's been a little over a week and a half since my last update. I'm hoping to do my next update in 2 more weeks once we are in the editor resolving issues with APB’s Social District’s map. Thanks, Matt

Hi all, I'm going to ask that you reserve judgment on STFU. We have a slew of information and articles coming out that will explain our "Web 2.5" hybrid approach. I'm tempted to remove the word "NFT" from anything connected with STFU, because they have such a bad reputation. STFU is a big experiment, but we're well aware of the market. It's would be folly to try and do what everyone else is doing. We have no interest in creating meaningless assets to make a quick buck. I am very interested in establishing clear ownership of content created by players so they can earn real revenue from it - if they want it. Stay tuned. Thanks, Matt

Hi everyone, I know it's been a while. I didn't get sick or die. I've just been buried with the studio and getting some work-for-hire projects shipped. Little Orbit is very close to announcing a couple new things we've been working on, but I'm way overdue for an update on APB. I finally unpinned the APB 2.0 thread, and I'm starting this one. Here is my first APB update in a long time to explain where we're headed. https://www.gamersfirst.com/apb/news/2022/8/5/message-from-matt Thanks, Matt

Glad you guys had fun!

Hey everyone, Sorry about the outage. We are back online. Thanks, Matt

Hi everyone, I hope you enjoyed the Thanksgiving holiday. We had some days off, but still made decent progress. At this point everything in Social is working except for collisions on the skinned meshes like statues. We started work on Financial this week as well. There are some issues with streamed assets that are getting loaded twice due to older and newer code running at the same time. We'll dive into that next week. For now, here's another video with progress. Working collisions are: Joker Ammo vending machines Mailboxes (specifically, the spot in Social where people gather most often) Marketplace Kiosks Static Meshes Streamed Terrain 2021-12-03 15-42-55.mp4 Thanks, Matt

Hi everyone, Apologies to the community. It's now been 45 days since my last update. I have been in Europe for the last 5 weeks, and my schedule was been incredibly busy during the trip. If I had to choose a single word to recap progress, it would be: "frustrating" Back on October 2nd, we had a big breakthrough, and I was feeling really good about next steps. You can see from the screenshots in that post that we had finally gotten collisions working on the sidewalk. That update was posted minutes after we got a build that worked. What we didn't realize at the time, was that if you stepped onto the grass, you immediately fell through the world again. Additionally, several different types of placed props and static meshes in the map also had collision problems. During the weeks that followed, we fell down a lot of rabbit holes attempting to find the issues. While we could have individually made those objects conform to the new collision system, doing so would have been wildly unsustainable. Once we start work on Financial, there are thousands of objects and checking everyone of the them would be a nightmare. Instead, we chose to take additional time to step through line by line in the collision code on the current live version of the game versus the engine upgrade. We need the conversion process to properly rebuild each map and all of the collisions in one pass. After a bit of digging, we were able to identify the core issue for terrain, so that is working properly now. As with all things APB, of course there are still issues. For some reason we have some oddball props and items that still aren't colliding properly. So the team is still working on restoring old 1.20 collision code to handle every specific case. Below is a video running around Social with nearly all of the collisions working. As I stated in the last update, ignore the lighting, we haven't turned that code back on yet, so everything is either fully lit or not lit at all. You can see once we get to the placed crates, those don't work. And the statues don't have collisions. As soon as we have those sorted out, we will reprocess all of Financial to see how that district does. apb-engine-update-2021-11-19.mp4 Thanks, Matt

The game is back online!

Hi everyone, Sorry about all the maintenance and downtime. We've been adding pieces to help with performance. I was notified about the downtime that started this morning, but it slipped past me and I didn't get a chance to post it. We're currently down for a bit longer. I'll post when the game comes back online. Thanks, Matt

Apologies. This one slipped past me. There was another resource increase scheduled that I forgot to post.

We are back online!

All everyone, We have just started some maintenance to add more resources for the game. Be back soon. Thanks, Matt

As previously confirmed. Servers are back online.

Hi everyone, We've taken the servers down for maintenance to address a couple issues that popped up. I don't yet have a time estimate, but we'll try to be quick about it. Thanks, Matt

Hi everyone, After two years, we are live! Please bear with us for any speed bumps. I have no idea how the servers will handle traffic. You can download the game here: https://www.gamersfirst.com/fallenearth/ We are working on the Steam version next. Thanks, Matt

We are knee deep in launch prep, but I wanted to drop in and thank everyone for their amazing response to this news! The community response is so much more than I expected. I will do my best. We are making good progress on other projects, so I'm looking forward to having more free time to interact with the community.

Hi everyone, It's been far too long between updates, but as some on you may have noticed I made a post on Twitter on the 2 year anniversary of taking down the Fallen Earth servers. After that I posted again in response to a tweet. While I should have posted that here to begin with, I am happy to confirm that "Fallen Earth Classic" is going to be a real thing. We are now at the very late stages of getting the servers back online, and as some of you on Discord have noticed, parts of the game are responding again. There will be an announcement soon on the day of launch, but before that goes out, I want to raise a couple discussion points here to set expectations properly. 1) I'm doing this in the spirit of fun and just getting the game back online. There were decisions that had to be made in order to pull this off. As we go through the rest of my bullets, just know that in each case I had to balance getting this back online in a reasonable amount of time versus the laundry list of things I'm sure players would have wanted. We did our best. My hope is that players will enjoy the challenge of re-conquering the wastelands from scratch while they wait for the next version of the game. 2) Steam doesn't work right now https://store.steampowered.com/app/113420/Fallen_Earth_Classic/ is now live. We are waiting on Valve to restore the old game. As soon as that happens, then we'll post an update here. 3) Previously, we had some major issues hosting the game. Nothing changed here. Those issues are wide ranging. They include being forced to host on old, out of support 32-bit Linux servers, attempting to work around a horribly bloated database with item duplicates everywhere, and expensive external hosting costs. This time around, we worked through each of those issues as we built the plan for bringing the game back online. We successfully recompiled under 64-bit and we're using new operating systems. We also completely wiped the server. (SIDE NOTE: I sometimes refer to this as "servers", but only because the physical implementation of Fallen Earth requires more than 16 different services spread across a number of boxes). That means that you wont have any previous progression - including characters, items, or G1 purchases. I know that's a bummer. Trust me. We looked at other ways to do this, but many of them would have required a lot more work, testing, and time. This is an experiment. We wont really know how the game is going to perform till we get a bunch of players online. Which leads me to... 4) There will be no paid options for the game. For various reasons, I don't want to monetize this version of the game. It is truly free-to-play (refer back to #1 above). I'm going to look at turning on the highest level subscription rewards for everyone, but there wont be any ability to purchase anything from the online store. With that out of the way, I look forward to seeing you guys online. Thanks, Matt

Hi everyone, Sorry for the delayed update. I know there's been a few weeks without an update. In general, I prefer to only share significant milestones with the community. As I previously mentioned, our focus has been on making sure we can load the converted 1.20 Social District in our 2.2 build. With a bit of software engineering work, we were finally able to get the Social district to load in the editor, though without proper lighting, shadowmaps - and importantly, collision did not work on the legacy feature data meshes. APB is an extremely large and complex game. It has taken extreme discipline in addressing specific parts of the game that do not work. When we go to fix a system, such as collisions, it's very easy to stray off-course and try and fix two or three other systems, only to have two more break in their place. By hyper-focusing on feature collision, we were able to track down where the system lost its data, and also add in collisions automatically on level conversion. As as a result of that effort, we now have collisions working in the map. The team would like to share with you a few screenshots of the 1.20 Social District in our 2.2 client build. Veterans of APB will notice the lack of palm trees. This is substantial progress. We have a minor crash to address, and then the team will start work on the other maps and missing functionality. Thanks, Matt

Hi everyone, It's been nearly a month since my last post, and for that I apologize. As I have mentioned previously, these updates are sometimes tricky to write when we are deep in the weeds fixing issues. It always feels like we are right at the cusp of a big break through, so it's tempting to wait "just one more week" till we have something more to show. I don't have something big to show - yet. But I do have significant progress to talk about. I'll admit, the last 4 weeks have been a slog. We've hit very nuanced issues surrounding exactly how to load the new 1.20 data we were trying to preserve. These issues crash our conversion process. Each time, we spend nearly a day converting packages to 2.2, hit a crash, go back to the 1.20 code, step thought side by side, find the problem, work out how to fix it, and then re-run the lengthy conversion till it crashes again. Some of that nuance comes from the fact we simply can't take all of the 1.20 data packages. There are specific APB 2.x screens or console-based packages that we need to preserve and merge. It seemed by the end of last week that we had finally addressed all the bugs, and that we might get a full map to load into the editor. Then we hit actual bad data in the 1.20 files. Empty meshes and broken references to missing 3D objects. The works. It is a mystery to me how APB 1.20 runs as well as it does. But each of these new problems would break as APB 2.2 tried to load those files. So this week we have been playing whack-a-mole with bad data and adding code to handle each case. Today, the entire APB 1.20 Social map loaded successfully in the editor. No palm trees. No odd foliage. Just the live version of Social including all of the feature data we need to move forward running in the APB 2.2 editor. It was a pretty awesome day. The team pushed hard to get Social up and running as an actual district in APB 2.2, but we hit an entirely new problem related to how levels are loaded at runtime. We will be addressing that first thing next week. Our next steps are: Get Social running at runtime so we can snap off a couple screenshots and run a benchmark against APB 1.20 Run the conversion process on Financial and run a benchmark against APB 1.20 Convert the rest of the maps. Re-bake all of the lighting with UE 3.5's Lightmass. Fix the main rendering shader so that lighting is applied properly for the special case light maps that Real Time Worlds created. Assuming the performance is where we want it, then we will get a small group of community testers onto our test servers. I know this has been a long process, but no one has attempted this new direction before. It's uncharted territory. As I have stated before, the original team at Reloaded felt it was impossible to port the game this way. But today we saw our first big sign of progress. Hang in there with us. I have lots of plans, but they all rest on making the game sustainable. Then we can dig into the fun stuff. Thanks, Matt

Hi everyone, We made some good progress on the loading of older 1.20 packages this week. We are still not able to load the Social district. However, I feel like we are very very close now. As a reminder, Feature Data was baked into the 1.20 maps for displaying static mesh actors and culling/rendering them under appropriate scenarios. That data was removed when the maps were upgraded to 2.0 (console) and 2.1 (Engine upgrade PC). While we have upgraded various packages regularly since the original work in 2014, the maps themselves were upgraded once way back then, and then permanently modified with new art. No attempt has been made to upgrade them from 1.20 to 2.x since then. Here is some of the work we finished on the Feature Data this week: We're now parsing and loading the feature data correctly in most scenarios. We added support for alignment of lightmap data under 1.20. Byte alignment is a big problem we recently uncovered. The original 1.20 maps are mostly 4 byte aligned, but most of the new engine is 16 byte aligned. Of course, we cannot actually use the lightmap data stored in the maps. But generating new lightmaps should be trivial, provided we re-save the package and discard of the old lightmap data. The same goes for shadow maps. We've made progress on aligning the shadowmap data, and will treat it the same as light map data. We will remove that data and re-save when we generate lighting. Another issue we have been sorting out are Prefabs. Prefabs are references in a map to instances of a static mesh. This was used heavily in 1.20 and removed for 2.x. Unfortunately, in some cases now the saved Prefab references in older maps point to invalid engine classes. This coming week we are going to work on cleaning up all the bad references. We're also utilizing a new test map that contains feature data in 1.20 to get the data ported over: The APB Login Level. That single map, which can be tested by simply loading the game, contains examples for most of the data we're porting over. Most of it is off-screen and can't be seen by player, but the map actually contains parts of the Financial District and is small enough so that we have sample data to test without running into red herrings. Once we can successfully load this map into 2.2, we will try porting over the Social district from 1.20 again. Once that district loads properly with collisions, we will attempt to load the Financial district from 1.20. That will let us do another internal benchmark, and continue the work from there. Currently, we don't see too many blockers in getting this task done, but it is worth acknowledging that none of this has been attempted before. There's bound to be ways it will break. Once again, there are no fun pictures to show the community yet. We're still aligning data and almost all of the work thus far is code work. I'm feeling good about this week. Ready to show more as soon as possible. Thanks, Matt