Jump to content

Recommended Posts

Hi everyone,


I've locked this thread similar to our original Engine Upgrade thread, so that I can post on a regular basis for people to follow.

This week I'm traveling to Europe in prep for Gamescom next week. Then I'm staying over another week to wrap up business there before I return in early September.
With that in mind, I'm setting a goal of posting every 2 weeks.

 

Right now, we’re progressing nicely through with APB 1.30 (also known as the 64-bit port of APB).

We’ve been monitoring feedback after my blogpost came out, and I get it. There is a lot to be frustrated with ‘yet another engine upgrade’ and the lack of tangible progress for players. 

Here is my update this week on exactly what actual work has been completed:

1)     We have finished porting most of the core engine functionality, excluding APB-specific gameplay code, over to 64-bit. This involved a lot of different changes, most of which are backports from the 2.x code, however, some are entirely new changes to the codebase so that 1.20’s functionality is fundamentally preserved. The engine code had thousands of errors and warnings that we had to work through. At this point, I'm assuming that any warnings are really errors that need to be resolved before we ship 1.30.

Here are specifics for some of what we had to do:

  • Set up projects to have a consistent struct member alignment across the entire project (backport from 2.x)
  • Ensure compatibility with the VS2017 toolchain.
  • Identify and set up versions of third-party libraries, such as Scaleform and PhysX, that will suit a 64-bit client build of the 1.20 codebase
  • Changing hard-coded, casted pointer types from implied 4-byte pointers to implied 8-byte pointers.
  • Clean up declarations of local variables hiding class variables.
  • Clean up declarations of local variables hiding other declared local variables.
  • Modify the codebase to support newer versions of specific middleware libraries and ensure functionality is not compromised


2)      We’re in the process of cleaning up the ‘core’ components of APB, such as the middleware made specific for APB. This is mostly complete but may need further adjustments.
 

3)      We are beginning to do the same conversion process for the Unreal Editor components.

 

Our next steps will be to finish converting the Unreal Editor components, and the APB-specific components. Once that process is done, we will attempt to launch the Unreal Engine Editor and attempt to get a map loaded in-editor. This was the same strategy we successfully implemented for the APB 2.x upgrade before we started beta testing and realized that the underlying implementation in 2.x was never going to perform very well.
 

This time, we have taken the right steps to ensure (crosses fingers) that the Editor process will go smoothly. My hope is that there will be minimal time from getting a map loading in-editor to having it running in our actual client/server setup. That will allow us to do some performance and stability testing on both the APB client and its district servers. I'll be looking for opportunities to share ‘client havoc’ as it comes up, as that seems to be a community favorite.

It's been a little over a week and a half since my last update.

I'm hoping to do my next update in 2 more weeks once we are in the editor resolving issues with APB’s Social District’s map.

 

Thanks,

Matt

  • Like 29
  • Thanks 14

Share this post


Link to post
Share on other sites

Hi everyone,

It’s been exactly two weeks since our last update.

Our progress continues with APB 1.30, and I promised to give more frequent updates on a bi-monthly schedule. My hope is that keeping that schedule will provide some insight into how the team is coming along.


Firstly, we have cleaned up all of APB’s dependency projects to compile under 64-bit. For the most part, we tracked down the appropriate libraries that would work with 1.20’s codebase out of the box in 64-bit, and in some cases, we modified existing libraries ourselves to fit the older version of Unreal. Between middleware integrations (such as Scaleform and Wwise), a rewrite of some 32-bit specific optimizations and texture utility code written in assembly, APB’s custom Music Studio and Web Browser libraries, Windows API support, and APB’s User Interface and Editor modifications, there has been plenty of work completed.

A total of about 3,000 warnings and errors have been resolved to date.

Next, there is only one more project to finish converting to 64-bit – the gameplay code for APB itself.

After the entire codebase compiles without warning or error, we will have to resolve issues that may arise from the process of linking the projects together. And then we can fire up the Unreal Editor and load a district. From there, we expect to find problems with serialized package structure alignment similar to what we encountered with the 2.x upgrade. Since most of these issues were already resolved in 2.2, I'm hoping we can reuse that work to do an ‘in-place’ upgrade of APB 1.20’s serialized content packages to support 64-bit architecture.

Even though we missed our previous mark of wanting to be in editor attempting to load packages by the next two-week update, we still completed a significant amount of work in a small amount of time, and we have not encountered any unknowns that we haven’t overcome before.

I’m hoping that we will be in-editor resolving issues with APB’s Social District map for the next two week update.


Thanks,
Matt

  • Like 32
  • Thanks 11

Share this post


Link to post
Share on other sites

Hi everyone,

 

It’s been another two weeks since my last update.
Here’s another update on our schedule.

We did pretty well.

 

At this stage, every project compiles and links under 64-bit without warnings or errors!
This is a significant milestone, as it means most of our time will no longer be spent merging in code, but instead debugging and getting the editor, game, and servers running.


We are now at the ‘get the editor running’ stage of the build process.
This involves resolving startup / initialization issues with dynamic link libraries, initialization, and instantiation of core engine components.

Then, we will ensure that we can rebuild and regenerate APB: Reloaded’s Unreal scripts.

After that, we will deal with existing disk-serialized package misalignments that would prevent maps from loading.


So right now, the focus is on instantiation of core engine components.


When we have all the items above sorted, then we will be able to attempt to load the Social District map.


Thanks,
Matt

  • Like 18
  • Thanks 7

Share this post


Link to post
Share on other sites

Hi everyone,

It's that time again - we have reached another two weeks since my last update. 

We are still at the ‘get the editor running’ stage of the build process.

The script generation commandlet needed some unexpected additional work.
We identified several new issues while building the scripts mostly related to serialization of classes. 

SIDEBAR "What is Serialization?"


I don't like using jargon that isn't easily understood by everyone, so since this update is pretty short, I'll explain what "Serialization" is. At a high level serialization is the concept of translating structured memory into a format that allow us to store it - more specifically APB data is stored in files on disk. Structured memory is another way of saying data structures or in-game objects. These are things like 3D mesh data, Gun stats, or data that controls how parts of a district show up in a mission. The critical (and exhausting) part of serialization is making sure that each member of a data structure is translated at the proper size into a format that can be reversed back into memory giving us the same original value. There are a lot of different reasons why sizes can be problematic or truncate data such as 32-bit -> 64-bit conversions that might have smaller or larger definitions of a memory block. This is made worse by the fact that many parts of APB had hard coded block sizes set at the top of the data structure. So we have to go in and find every one of them to make sure they are all handled the same.


END SIDEBAR


And of course... there are lots and lots of data structures for APB (and Unreal).


To address some of the issues we found, we spent some additional time pulling across relevant code sections from the 2.x upgrade back to 1.30.

Our plan has not changed, but this process of pulling across code is still going so that serialization works properly.
Then we can rebuild and regenerate APB: Reloaded’s Unreal scripts.. and then the editor will load.

Our current benchmark is still to get APB’s Social District map loading.

Thanks,
Matt

  • Like 21
  • Thanks 8

Share this post


Link to post
Share on other sites

Hi everyone,

Tuesday is here, and it's time for another update.

 

We’ve completed the work to get the editor’s script generation commandlet working.
 

We’re in the process of making the UnrealScript ‘source’ files match the headers we’ve manually modified in various places of the engine so that 64-bit support will work. This process is honestly more complicated than it needs to be, but that's just the way it is working with this version of Unreal. Essentially, it's a chicken or egg problem. Unreal compiles scripts to generate mores source code. But the old scripts wont compile with the new version of our 64-bit source because they are incompatible. So we have to create all source code that would normally be generated to get the source to compile. Then we can process all the scripts which will replace our manually generated code with the correct version.

It's a complicated process filled with potential ‘rabbit holes’, but at least it's mostly straightforward and mostly grunt engineering work.
 

We’ve also completed the work to regenerate the “Core” project’s scripts.


As of Friday, we ran into an issue parsing the source version of the script themselves that we’re investigating that currently results in a crash. Unfortunately, due to the age of the engine and the way older Unreal outputs text to logfiles, this has been a particularly painful issue as logging doesn't finish dumping once a crash occurs. So there's no help there. We’re currently in the process of identifying that crash, using a combination of three elements – working copies of the codebase that function as intended, debugger output messages, and our gut intuition on what the problem may be (insert all the community jokes here about how bad our gut intuition is).

Once this issue is resolved, we will likely have to fix the rest of the project’s UnrealScript package compilation. As I mentioned above, these UnrealScript files will be processed to generate native c++ source code that needs to match the files we’ve manually merged into the 1.30 codebase. It's a crucial step so we can continue forward and launch the editor.

We’re hoping to be done with the UnrealScript portion of 1.30 by the end of the month.

After that, we can begin work on loading existing content packages.


Our current benchmark is still to get APB’s Social District map loading.

Thanks,
Matt

  • Like 22
  • Thanks 9

Share this post


Link to post
Share on other sites

Hi everyone,

 

It’s a very spooky Tuesday in San Paro, and with the event comes another update on the progress on 1.30!


We’re still working on crashes since the last update.


We’re in the process of narrowing those crashes down and have a few leads on the root cause.


Unfortunately, this crash is about exciting to share with you guys as it is to fix, and it's also a blocking crash. The rest of the 64-bit upgrade is dependent on resolving this crash.
 

For the moment, I have our most senior engineer working through the issue. The nature of the crash itself means that we need to complete the fix before the rest of the project works. Allocating multiple team members to an engineering task like this does not speed up the work in question. Other APB team members are working on creating new rewards and enabling the Halloween event, which is scheduled to go live later this week, while the team allocated to working on 1.30 continues their work.
 

We’re still progressing.

Once we get over this irritating hurdle, we will complete the rest of the tasks necessary to get the Editor loading.

 

Still hoping that once we have the Editor loading, the rest of the components should ‘just work’, due to the nature in which we have been careful while upgrading the engine.

NOTE: I've received a community letter from a group of the players that I'm working on addressing in a much larger post. We're working through the strategy of possibly splitting each section into separate forum threads that the community can weigh in on, and I'll do a live AMA as part of my response. Stay tuned.

Thanks,
Matt

  • Like 16
  • Thanks 10

Share this post


Link to post
Share on other sites

Hi everyone,

 

The last couple weeks had unfortunate timing as several of our staff had vacation time scheduled, and we were blocked on a fairly tough crash.

 

However, everyone is back, and the team has resolved the blocking crash in 1.30 as of Monday of this week.
 

We’re continuing progress on the remainder of 1.30’s UnrealScript changes.

We hope that we can quickly move past these changes to loading the Editor and working to ensure existing packages load correctly.

 

Other members of the team aren't back till tomorrow, and there was a problem with removing Halloween, so we will be extending that event one more week. 
 

As mentioned in the previous update, I'm still working through a series of forum threads and scheduling a Live AMA with the community to walk through the community letter from several concerned APB community members. We’ll be sure to post information regarding that AMA so you can tune in and participate.

Thanks,
Matt

  • Like 24
  • Thanks 10

Share this post


Link to post
Share on other sites

Hi everyone,

 

It’s been another 2 weeks, and it’s time for an update.

 

We’re progressing through UnrealScript crashes. We’ve managed to complete script serialization work for all non-APB packages, and now have moved to APB-specific UnrealScript. This is great progress, and we managed to get as far as trying to load an Audio package as part of serialization this week.

 

The team ran into a 32-bit/64-bit alignment issue with the Audio package. We believe we have a grasp on the issue now, and it should be addressed shortly so we can keep moving forward. UnrealScript compiling and serialization is required to get into the Editor. The game primarily runs off "packages" which contain all the assets and scripts. But since the UnrealScript serialization process includes all of the package serialization, it’s my hope that the team will not need to spend as much time getting other types of packages loading.

 

Also, I'm running a bit behind on the community letter. I promise I'm not avoiding it. I just ran out of time before I left for vacation. (It was important to me to keep our cadence of updates, so I took time away from my break to post this.) I am hoping to get the community letter threads up as soon as I'm back in early December. Then we can do the AMA after everyone has had a chance to engage on the threads.

Thanks,
Matt

  • Like 23
  • Thanks 11

Share this post


Link to post
Share on other sites
Guest
This topic is now closed to further replies.

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...