MattScott

Hi there, We have been struggling with issues at the provider in Europe. We are rebooting servers (again) to try and clear the issue. Thanks, Matt

Hi all, We are restarting districts to fix the lag. Thanks, Matt

Hi all, This was a bug that we patched early on Thursday. For anyone stuck on pumpkins, please open a ticket with Support. I am having the team check these immediately to allow players to continue. Thanks, Matt

Hi all, We've been reading this feedback, and there were some good lessons learned on Halloween. In the future, we are looking at producing more general designs that have more re-usability. Thanks, Matt

Hi everyone, These are turning into Saturday updates, since that seems to be time when I can sit down and recap the work from the week. Right now, our #1 priority is performance. Performance for the Engine Upgrade was pretty good prior to when we merged all the code and assets from Live. After that, our performance has really struggled. For this post, I'm going to breakdown how performance is measured by our team, and what we're looking at. First, we spent this week finishing off items from above, and re-writing/improving the statistics that we collect in-game. This took significant effort to revamp old code and measure specific parts of the rendering and game loops so we can see where most of the time is spent. We also synced this new code between APB 1.20 (Unreal 3) and APB 2.1 (Unreal 3.5), and we created a new export process to chart performance in Excel. Second, we took this new stats engine and ran a series of test runs that isolate as many variables as possible. For instance, we tracked performance in Financial at midnight across APB 1.20 and APB 2.1. We use night time, because the game generally runs worse at night due to the extra lights that turn on. Each of us used different test machines, but mine was on the less powerful side with a 4 core i7-7700HQ @ 2.80GHz, an Nvidia GTX 1050, and 32GB ram. For the test we ran at the highest settings but used 1024x768 windowed mode to generate a lot of windows background interference. We also run no vsync and no smooth frame rate to get the highest rates possible. We measure a game's performance based on the low, high, and average frame time in milliseconds. Here are some common frame time to frame rate benchmarks values. 10ms frame time = 100FPS 16ms frame time = 60FPS 25ms frame time = 40FPS 40ms frame time = 25FPS Unreal 3 has always run the Render thread and Game Loop thread in parallel, but each process runs its tasks one at a time in sequence. The game's total frame time is the overlapping time from the Render thread and the Game Loop thread per frame. The Render Thread is mostly work done on your graphics card. The Game Loop thread is all the logic and keeping track of everything else including world state, physics, and player state. The Game Loop is dominated heavily by time spent ticking (processing) each relevant thing on the screen to make sure it's updated. All of this work resulted in a series of benchmarks that all showed the same general result. Live (Unreal 3.0 / APB 1.20) Financial at Midnight on my test machine. Peak performance = 10ms (100FPS) Low performance = 19ms (50FPS) *There are some bad hitches in Live that we throw out Average frame time = 12ms (83FPS) Average Rendering frame time = 12ms Average Game Loop frame time = 12ms Engine Upgrade (Unreal 3.5 / APB 2.1) Financial at Midnight on my test machine. Peak performance = 10ms (100FPS) Low performance = 19ms (50FPS) Average performance = 15ms (66FPS) Average Rendering frame time = 8.5ms Average Game Loop frame time = 15ms Right away, we can see that the Peak and Low performance of the game are the same. But there is a 3ms frame time / 17FPS difference in the Average frame time. Where APB 1.20 is more balanced between Rendering and Game Loop, we see a lopsided workload in APB 2.1. The Rendering thread is quite fast, but the entire frame time is held back by how long the Game Loop takes to run. For the rest of the week, we iterated on this process to improve our stat collection system and dig deeper into specific chunks of the Game Loop so that we could directly compare the performance of those smaller sections between Live and the Engine Upgrade. We were successful in identifying 2 Game Loop sections (out of 35) where APB 2.1 ran a total of 4.5ms slower than APB 1.20. We are now digging further into those sections to create more detailed stats to help further breakdown performance until we can define a fix to the performance difference. I'll go on record saying a couple things. The performance in APB 2.1 is already significantly better than what is running on consoles right now (APB 2.0). We will be finishing console blockers as soon as possible and submitting so the console players can get a much better experience. If I felt the performance was good enough right now, then I would be opening the Beta and putting this on public OTW. We don't have any other major blockers. The art issues and minor bugs can all be addressed after we let players in to test. Thanks, Matt

Hi all, Definitely getting caught up. 464 unanswered tickets 676 total tickets Responding to tickets created on 9/28, so we are running at around 3 weeks. If your ticket is older than September 28th, and we have not reached out, then I recommend updating your ticket and asking for status. Thanks, Matt

Hi all, We have been suffering from small, but consistent outages with our network provider in Europe for the last two to three weeks. Today we hit a significant outage in North America. These are not attacks against us. They just seem to be issues with the network. The team is working with our provider to get this fixed as soon as possible. Thanks, Matt

Hi all, Sorry for the miscommunication. I was traveling all weekend, and didn’t get a chance to connect with my network team. It looks like we got our time zones crossed. At 12:00 UTC, the Fallen Earth servers went offline. We spent all of our effort on the Last Stand event, so there was no time to make a cool countdown or other last moment. Again, I want to thank all the players who came back to say goodbye. We will be posting updates on the reboot here. Sincerely, Matt

Hi everyone, I am traveling right now, so I missed yesterday's update. I'm going to summarize the week here, and then modify the main post. Performance updates - this has been a little frustrating. We've seen side by side comps where the new engine outperforms or matches Live. But right now, the build being tested by SPCT is performing terrible compared to Live. We spent part of the week rebuilding our stat collecting system to gather deeper data related to the game loop. Various disconnect crashes - These have all been fixed (for the moment). Texture streaming bug - We addressed a couple more issues, and I'm treating this as finished. NPC / Living city character lighting issue - This is fixed but I have a couple non-essential changes I want before go live. Character customization saving is broken - This is fixed and needs to be deployed with the next build. Streetlamps in Financial - This turned into a huge rabbit hole, but we managed to get it all fixed. We need to re-run lighting for Financial before the next build goes up. This process runs overnight in the office and uses all the available computers on the network, and even then it still takes more than 8 hours to complete. We found a lot of new bugs that have been logged, but I don't have access to the list at the moment. I'll try to rework the list in the main post on my next update. Thanks, Matt

Hi all, I missed this yesterday. So the stats as of 10/11 were: 603 unanswered tickets 807 total tickets Responding to tickets created on 9/14, so we are running at around 27 days. If your ticket is older than September 14th, and we have not reached out, then I recommend updating your ticket and asking for status. Thanks, Matt

Hi all, We are about to start transitioning our architecture to support local districts and the world consolidation. We are going to do our best to eliminate as much down time as possible, but this is another big server transition. Our estimate is that we can accomplish this over the next couple weeks. When we're done, we should have districts in LA and NY for North America, which will be our first pilot of this new structure. Thanks, Matt

These changes have been made for the male harnesses. Just waiting on the female harnesses. We were very conservative about the angle of the shotguns for other reasons. The new angled version requires a different rig to make sure we avoid as much clipping as possible.

Hi everyone, The entire team has been crunching for September on the Engine Upgrade (including me), so I haven't had much time to blog lately. However, I have seen a lot of requests for updates, so I thought I would try something a little different. As I said on the latest Q&A, we're done merging all the content and code. We are just fixing bugs now. So below I'm going to list the exact bugs that we need to fix before we do the public OTW test. You'll notice we're not quite at the stage of fixing actual gameplay or collision issues yet. I'll try updating this thread each Friday to clear or add bugs as we move forward. All platform issues: Performance benchmarks - We've made a ton of changes recently, so we need all new benchmarks. From our earlier work, we feel the game is running well in terms of multithreaded rendering. The render threads are out performing the game thread. So all of our latest optimizations have been surrounding ways to improve core game logic. STATUS: Ongoing Various disconnect crashes - This only happens in "final" builds that we release to players. There is a crash in the game whenever you get disconnected from a server for any reason. This causes the client to flag itself for a full file repair, which is very annoying for testers. STATUS: In progress Level of Detail issue - This was extremely noticeable in the video that got released. Buildings popping in horribly. Trees disappearing. Etc. Turns out there was a tremendous amount of custom code in Live used to dynamically morph between various building LODs and to support individually lit windows at night. Reloaded tossed all this original code because none of it was supported under Unreal 3.5, and they just adopted the stock LOD system. Obviously this left many bugs. The team was able to pull in all the code we needed from the old engine and combine it with the new Unreal 3.5 LOD system. STATUS: Mostly fixed / Window lights on LOD0 at night don't work in "final" builds for players Texture streaming bug - Even though this isn't a crash bug, it has been our #1 issue. Texture streaming is a unique system RTW wrote to prioritize textures based on various methods, so that when the player's machine or console runs out of texture memory, the game can downrez textures that aren't used very much (or aren't very noticable). This is a lot harder than it sounds, and there are many different methods used (texture size rendered to the screen, distance from the furthest object using the texture, special flags for signage and other highly detailed textures, etc). For the most part, this is working MUCH better now. The game looks amazing. There are a bunch of signs in the districts that need to get special texture streaming flags, and in some cases we see oddball textures get downrezzed that shouldn't. STATUS: Mostly fixed NPC / Living city character lighting issues - This was very weird. We started noticing that sometimes many of the pedestrians were heavily under lit with a weird orange lighting that didn't match the scene. We've been chasing this for a bit, and found that the code for reflective lights was broken. The team was able to determine that reflective light values were being initialized based on whatever main direction light was active (or inactive) in the scene when they loaded into the district. This is being fixed so that we can control the reflective light values from our Time of Day data. STATUS: In progress / Mostly fixed Character customization saving is broken - There's a database change necessary in our external SPCT testing environment. The next push should fix it. STATUS: In progress Color Lookup Table grading is broken - This is a general post processing feature in Unreal where artists can apply color LUT files to modify the look of the scene. Right now, we're seeing some high thresholding / banding when LUTs are used. STATUS: Removed so the artifact doesn't show up / Awaiting dev resource Time of Day color values are broken - The lighting model for Unreal 3.5 is completely different than Unreal 3.0. As such, all of the values used to handle colors, depth of field, bloom, etc for midnight, dawn, noon, evening and night time need to be polished up. STATUS: In progress Streetlamps in Financial - It looks like an artist at Reloaded swapped all the spotlights to point lights for any lighting that comes from streetlamps in the district. In order to create the light halo on the sidewalk, they had to make the radius quite large. This means when the streetlamp is next to a building there is a bright patch where the light is affecting the wall. The team went back through and swapped back in the spotlights, but we have a bug in the prefab that needs our Financial map rebuilt. STATUS: Fixed but 2nd issue needs fixing Building feature LOD issue - We have a visual artifact because the rendering of mission features on buildings is not synced to the building LOD code. This means that for a small period of time we sometimes see holes in buildings where the features should be. STATUS: Awaiting dev resource We have submitted two console builds, and gotten kicked back for defects. Right now we're 94% compliant with the necessary checks that both Microsoft and Sony require to approve a title. Xbox One: 21 total must fix bugs (hightlights below) Title terminates on disconnecting the network cable anywhere in the title Null Characters are displayed on launching the title in any supported language other than English Background art and MAP fails to render on Xbox One X kit Title terminates on launching the tile in an Xbox One X kit The title crashes on suspending the title during the splash screen Title crashes in keeping the console idle for more than 20 minutes on the Music player Game Event limitation popup is displayed during gameplay SpaceBar is displayed for jump in Tutorial during gameplay Unnecessary Resolution option is displayed in Video Options Title crashes on adding friends from inside the game Title displays a black screen for more than 12 seconds right after the initial splash screen Title displays unnecessary button call out during gameplay Title allows the muted user to communicate through text during online gameplay Title allows the blocked user to send Group Invite User is unable to dismiss the interact menu by pressing B button after interacting with another user Title randomly crashes during normal gameplay The achievement 'Olympic Contender' fails to unlock on meeting its criteria User is unable to scroll thorough the description using RS in descriptions Title does not provide an option to invite another user to the game PlayStation 4: 12 total must fix bugs (highlights below) The term 'Sony Entertainment Network account' is displayed in the title screen Application provides 'Resolution' setting in the 'Video' tab Application soft locks upon selecting a user from the ‘Search’ tab of 'GROUP & FRIENDS' Gameplay - Incorrect naming convention is displayed in ‘Tutorial’ Incomplete online ID is displayed in the ‘Friends’ tab in ‘GROUP & FRIENDS’ Application fails to transition to a mode which supports play together User B fails to join the User A's session using 'Join' feature Application crashes on exploding a bunch of cars during gameplay Application incorrectly displays the SIE-specified text in the ToS Gameplay - Incorrect naming convention is displayed in 'Drive the vehicle 500 meters' tutorial Thanks, Matt

Hi everyone, I'm posting early today, due to meetings later. Big push this week. I think we're through a large chunk of the Nekrova players who had migration issues. There are now 905 total tickets and 674 tickets that have not yet had a response. That's a big drop from last week. Today we are answering tickets from September 9th, which means that we are at 25 day response time, which is nearly a week lower than last week. If your ticket is older than September 9th, and we have not reached out, then I recommend updating your ticket and asking for status. Thanks, Matt

Hi everyone, We're going to pause this effort for a number of different reasons: 1) Most importantly, our email newsletter system is being rebuilt, and we have not been able to properly message individual accounts that are in danger of losing their names. It's critical we give everyone enough lead time to respond and login. 2) Also I was made aware of a large number of players who are planning on griefing/squatting on released names tomorrow. I feel like this is simply going to create the same problem, and since players can create endless emails to register endless new accounts - we can't restrict how many names they can claim. 3) I had made the core assumption that we needed to release these names in order to facilitate a smoother consolidation of Jericho and Citadel, but that strategy has also been changed. The end result will be the same, but we have come up with a new way of identifying players that will eliminate conflicts. In summary, the name release WILL HAPPEN. For those that bought name changes, we're not that far out. I just need to think on how to do it in a way that wont get abused. Thanks, Matt

Hi all, I should have mentioned that Shotguns are the next major category we are tackling. I'll amend my post to reflect that. Thanks, Matt

Hi there, For the OP, I can't know how frustrating it is to wait that long and get a copy/paste response. I'm actually frustrated reading this, because we have internal standards that are designed to prevent agents from do this. However, I will say that often for griefing, hacker, cheat, exploit tickets, the only response they can give is an "acknowledgement of receipt". The team has been trained how to research these cases and then properly escalate specific data points to Easy Anti Cheat. But from there, we rarely get any feedback from EAC until there is a ban wave. In other words, it's not feasible for us to track all these types of tickets and respond when action by our anti-cheat has been taken. Per the OP's request, I'm locking this thread. Thanks, Matt

Hi all, After a couple weeks of investigation, we're changing strategies on this effort. I'll be posting separately on the reasons, the new timeline and the approach. Thanks, Matt

Hi all, First, this week we are going to push the older weapon balance changes to LIVE. We aren't done yet. We will be updating Prototype districts with new changes for other weapon categories soon. In making this post and reviewing all the changes, it appears we pushed RSA and ACT 44 changes to Prototype Districts, but the stat changes got missed in our patch notes. I recognize players might have missed these, but since they are very minor buffs, I have approved for them to go live as well. For the patch this week, we will be including the following changes: Low-Yield Fragmentation Grenade: Explosion Radius 700 -> 550 Health Damage 575 -> 515 Stamina Damage 365 -> 295 Hard Damage 435 -> 376 OCA-EW 626 (And varieties): Fire interval from 0.092 to 0.098 seconds, Accuracy Recovery from 5.675 to 5.25 Joker RFP-9 (applies to the base stats of RFP variants): Health Damage: 133 -> 110 Magazine Capacity: 21 -> 24 Ammo Pool Capacity: 105 -> 120 Radius at 10m: 10 -> 20 Min Damage Range: 25m -> 15m Marksmanship Modifier: 0.64 -> 0.32 ACT 44: Decrease Marksman Modifier from 0.63 to 0.55 Colby RSA: Decrease Marksman Modifier from 0.50 to 0.40 I want to thank the players who participated with feedback and iterations. Second, I want to discuss the Joker RFP-9 "Fang". The base Joker RFP-9 is designed to be a close to medium ranged weapon, mimicking the OBIR in design while being held back in range. However we have noticed that the 'Fang' variety of the Joker RFP-9 is currently dominating both medium and long range, due to the Improved Rifling 3 modification. The blooming of the Joker RFP-9 makes the drawback of Improved Rifling largely insignificant, leaving the gun in a very overpowered state. As a result, a lot of guns designed specifically for long range sit in the Fang's shadow. It has become the go-to secondary weapon for anyone playing a close range primary. If all of that wasn't bad enough, this gun is only available behind a paywall, which makes it purely pay to win. We have been making a strong effort to balance the P2W aspects of the game, and unfortunately the Fang is one of those outliers that needs to be addressed. For the record, I really hate changing a gun that people have paid for, even though I recognize we have already been forced to do this. Just know that we take it seriously. I would love to see the new Colby RSA and ACT 44 come back into the spotlight for long range secondary weapon engagements. In order to do this it is important to alter the Fang so that it is no longer dominant without breaking the base gun, which also just went through a lot of re-balance. We do not wish to change how the gun is played, but rather fine tune its role. With all of that in mind, I'm going to go on record saying that IR3 on the Fang sits at the heart of the problem we are seeing. It is simply a bad design decision. It's worth mentioning that we can't do mod changes in the Prototype Districts, because unlike stat changes, we can't change mods per district. The recommendation from design is to swap the Fang's mod from Improved Rifling 3 to Hunting Sight 3. Improved Rifling 3 increases the range while also increasing bloom gain. Hunting Sight 3 greatly improves accuracy in marksmanship mode. We can always tweak the base RFP once the mod is changed. But since we can't test this change without actually pushing it live, let's discuss it here. Thanks, Matt Edit: Shotguns are the next major category we are tackling.

Hi all, We took the servers down for about 20 minutes to make a quick patch to the event. Was back up very quickly earlier. Thanks, Matt

Hi everyone, We are still pushing hard to clear the merge tickets and catch up. There are now 957 total tickets and 768 tickets that have not yet had a response, which continue to come down from last time. Today we are answering tickets from August 26th, which means that we are 1 day over a month response time. If your ticket is older than August 26th, and we have not reached out, then I recommend updating your ticket and asking for status. Thanks, Matt

Hi all, After weeks of testing and tweaking, our final event for this version of Fallen Earth is ready. The event will start tomorrow on September 27th and end on October 14th when we will be closing this chapter of the game's history and turning off the servers. We hope you will spread the word and organize with friends to go on one last adventure (for a while) and say goodbye. You can find details here: https://www.gamersfirst.com/fallenearth/news/2019/9/24/the-last-stand We look forward to seeing you in the Wasteland. Thanks, Matt

Hi all, The devs are looking at this. No need to open a ticket. If you made a purchase, we’ll get you sorted. Thanks, Matt

Hi all, We've hit a snag with the updater service that builds the patches. Obviously this infrastructure hasn't been used in a long time. The team is continuing to work on this through the weekend, and I'll give updates as we move along. At this point it looks like we stand a good chance of missing the 9/22 with this event. Regardless, we're going to run it anyway as soon as it's ready. I appreciate everyone's patience, and I hope everyone enjoys the various player-run anniversary activities tomorrow. Thanks, Matt

Hi all, Commander has been applied to everyone. Thanks, Matt