MattScott

Hi everyone, Jericho Outage Just a quick note to let you know that we are currently experienced a network outage with our provider that is affecting Jericho. It is not a DDoS, and we are still investigating with them. New RIOT Changes We aren't putting a lot of focus on RIOT, but we did solicit feedback from the community, and I wanted to follow through on that with a significant number of changes to the mode based on that feedback. To be clear, if RIOT isn't your thing, I totally understand. New RIOT Weapon Skin Reward As part of the patch, we added more tiers and new skin. I recognize this isn't super compelling. I do feel like the Joker Ticket rewards are fairly valuable given the growing selection of permanent weapons in the Joker Store. Just to be upfront, there are going to be three Japanese skins in total for RIOT. Yakuza. Dragon. Koi. They were all finished some time ago, and we decided to save them so we could give RIOT players something while we finished off the Engine Upgrade. These are just meant to be fun rewards for those players that want them. You don't need to chase them if you don't want to. All three skins will be going onto ARMAS in a pack or possibly the Joker Store at a later date. Bad Math on tiers 6-10 of RIOT Here is a quick note from LORider, one of our designers on a mistake with the new Tiers. "Some of you have mentioned in great detail the issues regarding RIOT’s progression. Admittedly I did not take into account the time requirement for queues when finalizing the progression for the new tiers and as such the requirement to progress requires more time then intended. We are working on some back end changes that will more closely simulate a single queue system which will reduce the time required but until then I want to course correct on the pain of the wait times. In the next patch I will be lowering the contact threshold for levels 6-10 from a requirement of +17500 contact experience to +12500 contact experience instead. Any experience gained will be kept and reflected properly when we make this change. I am happy to receive feedback and hope to respond swiftly when able." And some higher resolution art elements created by the artist responsible for the these three skins. Thanks, Matt EDITED: I can't spell Koi. And added elements of the skins.

Hi there, Back in January 2018, G1 did a massive password reset across all players. Unfortunately this required many players to go through Support to get access again. Once we took over, my brand new Support team was flooded with these requests and our response times became very long. However, the team has been working hard and we have brought down our initial response time significantly. We are currently at 3 weeks right now, but I expect that to continue dropping. Thanks, Matt

It was a completely different company’s game.

Hi all, Just a quick note to say thank you to everyone who sent me latencyreports over the holiday. We are using that data to do more tuning on the solution. Thanks, Matt

Hi all, Jericho has not been as stable as I had hoped over the holiday. A couple large scale DDoS attacks did get through but we will be tuning systems to prevent future attacks. Beyond that one of the games adjacent to us at our provider got hit hard, and it took us down several times. We’ll continue to work on it. Thanks, Matt

Hi all, For anyone that is experiencing lag, please run /latencyreport in-game and then send the log to me over Private Message. I can confirm that the DDoS protection has been working since it was installed. It sounds like these issues may be legitimate server issues. I am hopeful that we can start back up working on server stability / speed, now that we can eliminate DDoS as a source of slowness. Thanks, Matt

Hi all, As of 8am we went live with the new Anti DDoS solution. The team has been monitoring all afternoon, and it’s looking good so far. Seeing some nice low latency just in time for the holiday weekend. For those US-based Jericho players who have been frustrated, I encourage you to give it a shot. This solution uses a combination of systems, many of which had to be installed at the edge network in front of our gateway and servers. It was originally scheduled to go live in April, but our provider kept delaying due to conflicts with some of the other tenants using the same network. Hopefully this is a good step in the right direction. Thanks, Matt

Hi everyone, Due to the holiday, I'm going to go ahead and post stats early this week. There are 751 total tickets (unsolved) which is our lowest amount to date. And there are 440 total tickets that have not been touched (contacted). Today we are answering tickets from June 11th, which means we are now at a 3 week response time. If your ticket is older than June 11th, and we have not reached out, then I recommend updating your ticket and asking for status. Thanks, Matt

Hi all, I think SPCT are all done showing off progress for the moment, but @rooq took the time to edit down a flythrough for everyone who is interested. You can see it here on the Little Orbit YouTube channel: I'm looking forward to using this thread as a "starting point" so that we can show more progress with lighting and post processing soon. Thanks, Matt

This month I spent some time to walk through Mission #2 and the process for building it. Mission #2 - The War Summit Cutscene #1 - The Secret Chamber We step into the secret chamber to find that all four delegates from the most powerful areas in Lasfaria have arrived and the meeting is already in progress. The Queen of Alionne looks frustrated. Things aren't going well. An argument has broken out between the delegate of Jelamond, a small country to the north and home to the School of Divine, and Lethanor, a territory currently under Alionne control on the border of Saxtel and home to the School of Mana. Sirena attempts to interrupt with her important scientific findings, but they ignore her. Brienze, the Horned former Emperor and delegate from Saxtel, slams his fist on the table to gain everyone’s attention. “The soil has claimed enough of our tribes.” Suddenly, a loud metallic bang spooks everyone. Their secret location has been compromised. The player rushes outside alongside Sirena and Brienze and are surprised by attacking bots and sinister Saxtel units that have invaded the city. Brienze quickly denies any connection with the attacking Saxtel units and jumps into the fray to protect the delegates. Do you trust him? End of Mission #2. Thanks, Matt

This month we spent some time to walk through Mission #1 and the process for building it. Mission #1 - The Bridges of Branholme Lasfaria is a turbulent world easily manipulated by sound magic. The Opus, or the First Song, was created thousands of years ago to help keep the world stable and more hospitable to the mortal races. This scene establishes Branholme, the capital city of the Alionne Kingdom, as the central location of this chapter where 77 years of war with the Saxtel Empire has taken its toll. The Alionne army is failing, and the country is suffering from unexplained earthquakes and tremors. Times are desperate. Cutscene #1 - The Branholme Opus Room After the intro cinematic and character customization, the game starts with a brief cutscene in the Opus Room in the School of Technology. This room lies at the heart of the school and is a large cylindrical space of immense power with a hovering ball of light and energy at its center and a series of large rings in place spinning around it. Maestro Greymore, the Head Master at the school, has hired the player's characters to travel to the edge of the city and escort Sirena, a reclusive Lore Master, back to the Opus Room safely. She has recently discovered critical information and needs to convince a gathering delegation to end the war. The player leads a small mercenary band called The Unsung. This group does not have loyalty to any of the warring factions and are relatively unknown, making them a perfect choice to help the Maestro; they can be trusted not to betray for political reasons, and they can operate covertly. Cutscene #2 - The outskirts of town The player finds Sirena just outside the city's walls at the Branholme Observatory. She wears the robes of a Lore Master (one of our Prestige Jobs) and wields a staff. Sirena thanks the player for their help escorting her, worried that it’s not safe travelling these days. The war has created lots of conflict and you never know someone’s allegiance or intentions. Suddenly Branholme guard bots approach Sirena and the player from the direction of the first bridge. The bridge is disabled behind them. These bots have a strange green magical aura circling around them, and they claim Sirena is trespassing and must be stopped. She explains that the city is normally guarded by these mechanical constructs, but something has turned them hostile, and the mission starts. Earlier this year, I walked through the process of our Vertical Slice. We started with the overall narrative, and identified 50 missions. Our level designers created sketches for each mission, and then started building out white boxes so we can play and test them. Once a white box is complete, we create a Concept Guide. Here is the Concept Guide for Mission #1. From there, we work with the concept team to flesh out a series of 2D paintings that match the layout of the white boxes. These concepts are used by the 3D team to build the final level. Post Mission Cutscene #1 Sirena and the player arrive safely at the School of Technology. It's been years since her return, and she circles the Opus Room in wonder, talking out loud. “You’ve heard about how we sing into the Opus stories about our world and our lives to preserve it. At this school, we use Technology but each other school is unique with their own form of magic - Mana, Divine, and Nature." She motions to the Cantors, old singers stationed around the room casting spells directly into the ball of light. "When they sing about our accomplishments, our mastery of magic, and our progress as a society, it strengthens the bonds of the Opus which helps hold Lasfaria together. The world literally depends on this act, keeping the energies of the Opus in harmony." Gravely, she turns to Greymore, “But something is wrong. I have long been researching the decline of the Opus. The tremors that break our land. Chaos is growing, and soon every nation will suffer as we have. For years we have only sung songs of death and destruction. It's time to end the conflict." Greymore presses Sirena to convince the gathering delegates from each nation of her findings. She agrees to go to the meeting but asks for the player to stand guard in case it erupts in violence. But first, the player needs to prepare. Introduction to the Hub At this point the player gets a brief introduction to the Player Hub where the player's units can rest, equip themselves, and work with other craftsmen to improve. End of Mission #1. Thanks, Matt

Hi there, This was an a fix to Fallen Earth. You mentioned RIOT, which was a new mode for APB. I’m a little confused. Thanks, Matt

The game is back online. We fixed a significant problem. Curious to see if it helps performance.

Hi all, We need to correct an issue in the database. Taking the game down real quick to reboot servers. Thanks, Matt

Bad math on my part. I used the number of packages converted instead of the number of packages remaining in the second portion. I have updated the proper numbers in the thread. ---- To give specifics, there are 1498 packages used in the game: 919 regular packages 537 map packages including dynamic layers to action districts for events and missions So far we have converted about ~84% of them, with the following packages remaining to be completed: 212 regular packages 24 map packages

Hi all, I should have made it clear that we are not done with the lighting or post processing. We did an initial pass on time of day and some basic lighting values just to course correct what was done for console. But the art team is now waiting for the content packages to get migrated and then we plan on doing a fairly big pass to adjust the look and feel. There are several excellent threads on the forums already that show off the older RTW look. I wont guarantee that we are going to end up exactly matching what was done before, but there is consensus on the team that we want to steer in that direction. Thanks, Matt

Hi all, Apologies for the outages last night. Our hosting provider for Jericho is doing some extensive site upgrades that include a number of features we should benefit from. This work was originally scheduled for April, and has been pushed out multiple times. We missed communicating this to you guys due to numerous rescheduling notices, but the maintenance time window ended at 15:00 UTC. My team will keep an eye on servers to see if there are any more outages. Apologies for the lack of notice. Thanks, Matt

Hi all, Patch notes are up for tomorrow's patch. We have a couple optimizations for RIOT, but the majority of bigger fixes are still being implemented. We want to see how these changes affect things. https://www.gamersfirst.com/apb/patch-notes The team is still testing changes such as: - Sound in the district - Improved player messaging - Lowering the minimum player threshold - Reworking the end stage - Possibly removing respawns in the end stage - Increasing the protection time of the Hazmat suit gained when respawning Thanks, Matt

Hi everyone, We are way overdue to show off screenshots, so I have decided this update is going to be a bit different. To start, I’m going to give a brief update of where we are on PC and console, and then (for better or worse) I’m going to let SPCT post some screenshots that they chose from the current build. All previously shown Engine Upgrade shots were real, but they were done under highly controlled conditions. No HUDS. Fly throughs. Etc. Today's post is an attempt to lift the curtain today and give a genuine look at where we really are. For PC, everything is now done except for migrating Unreal package content. To give specifics, there are 1498 packages used in the game: 919 regular packages 537 map packages including dynamic layers to action districts for events and missions So far we have converted about ~84% of them, with the following packages remaining to be completed: 212 regular packages 24 map packages EDIT: Corrected after someone pointed out my bad math. That leaves 249 packages to migrate, but these are all somewhat more difficult. Some of them wont be migrated because G1 or LO already made changes that weren't ever in 3.0. This includes reworked Financial and Waterfront textures and geometry. Some of the packages are for User Interfaces. These are tricky because we have a chunk of UIs in 3.5 already that have been glued up for console. And we have new UIs in 3.0 for the trading system and other areas. Anything we migrate from 3.0 will step on aspects of the 3.5 screens and work will need to be redone. I'll have a better sense of this effort once we get a little further. Some of the remaining packages have changes in both 3.5 and 3.0 and those will need to be hand merged. Again, I'll have a better sense of this effort once we get a little further. We still have plenty of bugs and crashes that need to be fixed, but we now have all the code, systems, and server work completed. Performance Benchmarking Performance testing of the Engine Upgrade is very important. Not many people would trade better looking graphics for worse performance. In APB's case, there are currently disk-based stutters and bad support for multiple CPU or GPU cores. One of our primary goals with the Engine Upgrade is to fully utilize modern hardware. DISCLAIMER: It is important to point out that the Engine Upgrade has an entirely different rendering pipeline - some good and some bad. Many players have seen what the console builds look like, and the performance on those devices is terrible. The starting point of the Engine Upgrade was these exact console versions just running on PC, so there has been a lot of work to do since we started. Unfortunately most of the work is very specialized and cannot be done by a huge team without tripping over each other. We have kept as many people dedicated to the upgrade as possible, but this work is filled with rabbit holes and pitfalls. Sometimes areas that appear simple to optimize end up using global data that touches many other areas of the code. I want to thank @SKay for all of his hard work leading the benchmarking effort to help us gauge progress in these areas. To make sure the test runs were valid, they had to be able to easily reproduceable as well as being able to give a proportionate representation of what a user might see during their time in APB. The game environments vary from claustrophobic to wide open, and the benchmarks needed to account for both. In the end, he chose a short car drive that would force the game to move quickly through as many different areas as possible under consistent conditions. The main goals of the benchmark are to insure that: The 3.5 performance would at least reach the same speeds (on average) as LIVE The main issues (stutters, low 1%' and .1% frame rates) were at least looked at and/or addressed New visual upgrades or changes didn't affect the overall experience The route above shows the benchmark run that was used in all cases. The test run as outlined above has open areas and enclosed areas in a single run that could be repeatedly reproduced for apples-to-apples comparisons to help identify progress with the Engine Upgrade. Starting at Darryl Kent, we take a left out and stay on the road passing right of Suji. Taking the next right and keeping the City Council building to our left, we take a right at the crossroads outside of Violet Prentiss and Wilson LeBoyce, following the road as it sweeps towards Javez and passing him on the right. We hold that road and turn right when it meets the river, coming back to end in front of Darryl Kent. Stats were captured with MSI Afterburner using RivaTuner Statistics Server running on Ultra settings 4K resolution. Here are the benchmark results: To explain the numbers for the graph: Max FPS is the highest framerate detected during the test. This in conjunction with the other metrics can determine if the game is able to push good framerates on the current hardware. Avg FPS is the average of all the frames that were taken during a run. This is used as a base metric to compare the other metrics in this list with. Min FPS is the lowest framerate detected. This is not necessarily the lowest framerate outright, rather the lowest framerate that could be normally detected through normal gameplay. 1% FPS is the average of the lowest 1% of frames that were taken during a run. This metric, when compared to the average, is used to determine if the PC is struggling to run the game, or if the game is struggling to drive a consistent frame rate. .1% FPS is the average of the lowest .1% of frames that were taken during a run. This metric is used to detect for stutters or freezes (if any) NOTE: The benchmarks do not included the non Multi Threaded renderer results. The state of the renderer was bad enough when we started that it was clearly well below LIVE and not acceptable. Here is a quick summary of our development timeline on the Engine Upgrade: May 2018, we chose to focus on infrastructure which took us a couple months to clear the cobwebs End of July I posted my first update on the Engine Upgrade with details around bringing on more devs for the Unification project November we rebuilt the Music Editor for 3.5 and the majority of external libraries had been upgraded just to get the build running Early February we had finally sorted out a number of crash bugs that allowed SPCT to get their first play tests on 3.5 March we had realized how bad the renderer was and had implemented most of Multi Threaded rendering And here is a walk through of the results working up from the bottom: The bottom shows our LIVE comparison which was taken in the March / April time frame. The first set of results above the bottom was from the initial Multi Threaded renderer implementation back in April. These results were very encouraging, however in-game lighting was completely broken in this build, so the results were skewed higher than any other build that had lighting working correctly. 806538 shows what the initial finished implementation of Multi Threaded rendering from May. This was our first good look at performance, but it revealed several areas in the renderer that needed work - mostly in the Culling and Level of Detail systems. Culling is what decides which elements on screen to render and which to ignore. Level of Detail (LOD) determines if we can show a lower resolution model based on distance. 809566 shows some early Culling work from late May, but the routine had severe graphical issues with objects not getting rendered properly. 810873 shows our current build that is rendering properly with Culling and LOD fixes. EDIT: We are not done with the lighting or post processing. I should have made that clear. We are waiting for the content packages to get migrated and then we plan on doing a fairly big pass to adjust the look and feel. EDIT: @rooq spent the time to edit down this video showing the current state of the game. I look forward to doing comparisons against this as we finalize lighting and post processing. For console, we have a 2nd round of bugs to fix. The initial builds did not pass QA (quality assurance). Several crash bugs were found, so we couldn’t submit to Microsoft or Sony. However many have already been fixed. XB1 is being tested internally, and it may be ready to resubmit. PS4 has a couple more crash bugs left. And now onto the screenshots. Each SPCT member is going to start by posting a single screenshot. Feel free to request areas / content you’d like to see, but just so we don't make this thread too long I've limited them to 3 shots total. Thanks, Matt

Hi everyone, Lots of posts / updates planned for today, but I don't want to forget to update the Customer Support status. As of today, there are 913 total tickets (unsolved) which continues to come down week by week. This is our first time significantly under 1000 total tickets in a very long while. And there are 662 total tickets that have not been touched (contacted), which is down from last week. Today we are answering tickets from May 24th, which means we are now at a 27 day response time. If your ticket is older than May 24th, and we have not reached out, then I recommend updating your ticket and asking for status. (NOTE: Updated the stats as of the actual End of Day Friday) Thanks, Matt

Hi everyone, First, many many (many many) players have asked why we "wasted" time doing RIOT, and I think that's a valid question. The mechanics for RIOT were specifically designed to give my team a chance to learn how the inner workings of APB fit together. I felt that we could implement this mode without taking a lot of dev resources, we could keep our focus on the Engine Upgrade, and that the experience would give us an excellent sandbox to make mistakes. If we messed up RIOT, we could simply set it aside with no lasting damage. If we mess up the main game, then it becomes much more difficult to fix. The design for RIOT also included a number of underlying systems that we need. Some of them are focused on design elements such as cleaning up the district blocks or tracking new aspects of play. Other systems focused on replacing Joker Mystery Boxes. It's clear that legislation will eventually make these illegal, so we need new ways of building value that our players can spend money on. Next, here are some interesting stats for the first week: - RIOT was played 4x more on Citadel than on Jericho due to low population issues starting the mode - Fight Club ran 2.2x more matches than RIOT during the same week on Jericho, but 92%+ of accounts playing FC also played RIOT - Fight Club ran 1.2x more matches than RIOT during the same week on Citadel, but we saw 1.5x more accounts in RIOT than FC - ~40% of the active player base during the week played RIOT at least once We are spending a lot of time going through all the feedback here on the forums to see what is feasible. There have been lots of great suggestions. There are clear issues with the mode, and the team is working on a relatively light weight pass of tweaks and fixes to make it more fun. Areas that need work are: The end-game, Sound, and respawns. Right now, the big decision I need to make is whether to: (1) take the mode down until it has seen major work or (2) add some more tiers along with another reward that will enable players to continue grinding Joker Tickets while we make changes. I'm not interested in clinging to or defending something that isn't working, but I still see some value in the mode itself - just perhaps not in the original way we intended. EDIT: I have two more updates planned, but ended up in meetings for the rest of the day, so I'll do those next week. We are not moving forward with any Battle Pass or Seasonal content till the kinks are worked out. Thanks, Matt

Hi all, It's been far too long since my last update, so I thought I would take a moment to walk through where we are at. Let's start with April and May - both very bad months for Fallen Earth. First, our data center migration caused lots of unscheduled downtime as we attempted to repair damaged databases. That issue led to the loss of data, which I am personally very sorry for. Second, in May we had an issue get out of hand that involved a number of FE GMs. We made some mistakes on our end, and they made mistakes on their end. I'm not looking to rehash what happened, and frankly I wish things had gone down differently. Unfortunately that situation escalated into several bans and a lot of toxicity on the Unofficial Fallen Earth Discord. I can appreciate how upset some members of the community were, and that they were working on incomplete information. I work hard to generally be available to players, but that doesn't mean I need to subject myself to the level of harassment present on that server. One final point before I move onto the update itself. Several members of the FE community have been vocal about their negative opinions of Little Orbit. However, I would encourage you to question their statements based on the following facts: - The game wasn't making enough money to pay for itself when we bought it, and it still doesn't on a regular monthly basis - I had every reason to shutdown the servers in May after the horrible problems in moving them, and I didn't - Instead, we spent many extra hours at enormous expense to fix things and doing our best to keep the game online The only thing that will turn around this game is actual progress. Fixing the servers. Updating the client. Adding new content. If these vocal members of the community want to discuss things, I am always available here, but there is zero benefit to lying about the progress of the game. With that out of the way, here is where we are: Client port Up until May, our focus had been to work on the porting the client. As I have stated before, the client-side 3D engine for the game created back in 2009 is completely proprietary. We started the porting process by migrating all of the art assets into Unity. This was complete a couple months ago. The Unity project now supports all the textures, buildings, props, characters, armor, and animations. The terrain in the game is a completely different system, so that doesn't work yet, and for clarity, none of the actual logic for moving around and interacting in the game client has been migrated. Server port In May it became clear that the FE servers are not in great shape. As I have mentioned before, it takes over 1 hour simply to reboot them. They require special build servers to compile, and a long dead version of Linux to run. With that in mind, we switched the plan in May to focus on porting the backend. This is getting 100% of our focus right now. Back in December I posted this update: That post attempted to break down the code in a way that we could use to provide benchmarks. Unfortunately there are no sexy screenshots that I can post showing off server work, but I can try to add more detail to help describe our progress. That original post described the 1.65MM of lines of code in the common libraries and in the server backend. The common libraries break into two categories: (1) server-side Game Engine code and (2) other libraries. At this point we have ported ALL of the other libraries, which means we have churned through more than 200K lines of code in a relatively short amount of time. To give further context, the game's back end is made up of 4 different database servers, 10 databases (some duplicated on each database server), and 19 unique services (many of which have multiple instances running). In total there are over 100 service instances that all coordinate to make the game work. Our initial target in May was to re-write the LoginServer service - 1 of the 19 unique services of the game. We chose it because it didn't touch any of the Game Engine code, and we felt we could port it in a way that would maintain 100% compatibility with the other legacy services. This will allow us to slowly replace each service as it is ready, so that players can start to see the benefits of the upgrade. The new code is 64-bit, can run on modern Operating Systems, and can take advantage of newer processor architectures. I am excited to report that we finished work on the LoginServer last night. There are some dependencies on a couple other services, so we haven't been able to test it yet, but that should start next week. We have started work on the MasterLoginServer now, and work on the remaining services should go a bit faster now that all of the other libraries are finished. However we are leaving the main GameServer for last since that will require porting over all the server-side game code. I know a lot of this sounds like mumbo jumbo. I wish I could fast forward time and get through this process a lot quicker, but it took years for Icarus to create their engine and more time to create Fallen Earth on top of that. Thanks, Matt

Hi there, You can always email support@gamersfirst.com to open a ticket. Make sure to explain your situation and call out that you are trying to get into Fallen Earth. Thanks, Matt

This is absolutely not the case. We are dealing with a networking issue related to a new DDoS protection system that has gone a bit haywire. The team hopes to have the connectivity and latency issues resolved soon. Thanks, Matt

Hi all, We are looking at this issue now. Thanks, Matt