MattScott

Hi everyone, Edit: 06/02/2018 It's now been about 3 weeks, and I want to take some time to speak directly to the Fallen Earth players. First, I know there have been lots of questions asked on the forums. I have spent some time reading many of these, and I found them intricate, well written, and virtually impossible for me to answer right now for reasons that I'll explain below. I also want to acknowledge the many players who have reached out to me offering help or detailed walkthrus of some of the issues plaguing the game. That kind of detail helps. I've been in-game a little messing around with specific mechanics to familiarize myself with some of what has been talked about, and I'll continue to get up to speed. Second, I don't have a long term plan or roadmap for the game yet. I'm still in the investigation phase. So instead I thought I would write a short post to talk about where we're at and what we have to work with, because I think that will guide how we proceed. The codebase for Fallen Earth is split into three parts: - The original Icarus engine and tool set used to build and maintain the world - The Frontend client / updater - The Backend servers Let's start with the Icarus Engine and tools. Unfortunately, there is some code missing here. We have one developer who has been slowly reverse engineering the missing pieces just so we can get the tools back up and running. There has been some progress, and we obviously have the output from the engine, which is what the servers operate on. So in theory the tools could be entirely replaced some day. But the editor itself doesn't work right now. Next the Frontend client. We have all the code, and it is theoretically possible for us to patch the game or the updater. However this code is significantly old. It requires tools that are more than a decade in age, installations of old libraries, and very customized windows installs to even build everything. It is a matter of time before current operating systems wont be able to run this client. We need to think ahead and start migrating to something much newer and maintained. This means a complete rewrite using the framework of what has been built as a roadmap. The challenge is that we also don't have any of the source art files. No 3DSMax files. No PSDs. No high rez textures at all. We have reached out to the original dev team to see if we can get these, but in their absence, we will need to create these files from the data that ships with the game. This is not at all ideal, since these files have all been compressed or down rezed to work. But it will at least be a starting point, and it will give us an idea of scope for the art, so we can assess how extensive it will be to rebuild. Lastly the Backend client. This is in the best shape. The developer who is currently assigned to the project has been able to piece everything together so that each server can be rebuilt and redeployed. I think this is where we can make the most short term improvement to the game by fixing bugs and exploits. Ultimately we will still want to migrate these systems into whatever the new framework is so they can be more easily maintained. Where do we go from here? Please don't read this as Little Orbit giving up, or even us being discouraged. I knew most of this before we took over GamersFirst. I just want to share where we are at, and that it's going to take some time to provide a clear roadmap of how we're going to rebuild. From there, we can look at the design and possible changes down the road. I am still looking to schedule some time to answer as many questions as I can with players. I'll have Lixil post that update when we get a little closer. Edit: 06/15/2018 This week was E3, so I didn't get a lot of time on Fallen Earth. We have been able to extract the compiled game issues from the main data file. We have been able to get about 40% through re-organizing all the C++ code. The main problem was that several core libraries were copied and included several times. It's impossible to know which version is the most up-to-date, but it's clear they aren't all the same. So we've started moving everything into a new repository with each library referenced and only included once. We still have to do this for all the server code and some of the front end code, but we were able to get some of the tools recompiling this way. Thanks, Matt

Hi all, Lots of discussion here. Obviously things got a little off the rails. I want to acknowledge the players who have had bad experiences with non-employee GMs in the past. This is something we will need to manage going forward. I think we've reached a solid understanding of what Lixil intended with this post and the reaction from players. I'm going to lock and unpin this now. Thanks, Matt

Hi everyone, I appreciate the OP's sentiment, and we are doing what we can with the existing tools in place. The next patch with BattlEye is very very close. I'm going to lock this thread, because I don't really see any further value in it. But I hope the OP will give us another chance down the road as things improve. Thanks, Matt

Hi everyone, I really enjoyed reading this thoughtful discussion about why players have stuck with APB. It's one of the first entirely non-toxic threads I've seen on the new forums that made me smile, because I got an inside view into the game from the perspective of some older players. Thanks to the OP for not only posting the topic, but then prompting more people to answer. And thanks to everyone who engaged to give honest and open answers. Matt

Hi guys, I am reading these. Lots of good questions here. Thanks, Matt

Hi everyone, I've seen a number of players who are frustrated at the extended delays with our support team. First, I want to acknowledge that we did send a cut/paste email to all the players who opened tickets in order to get unbanned. Many of them went to players who had been banned for cheating. Unfortunately some of the unban requests were for different reasons. It can't have felt very good to get a cut/paste email response that didn't even address your ticket. Second, it's clear that my staff was too small. We worked hard to get the new support portal up, but the number of tickets and types of issues far exceeded our capacity. Part of the problem is that my staff are still learning all the tools. They have three different systems they need to switch between in order to look up or correct issues. They are getting faster, but this is taking more time than I thought. Here are the current CS stats right now: - 1,689 total tickets have been created for APB since the new portal went up on 5/25. - 480 of them were players requesting unbans. Since we will be unbanning so many players after the next patch, we are not going to be responding to unban request until after that date. As I said above, if you requested an unban, you should have received an auto respond message. - 252 of them have been closed. - 957 are untouched. The vast majority of the current open tickets are players who can't access their accounts after G1 reset their passwords in January. These are time consuming tickets. My team takes a lot of extra precautions to double check many different data points in the account to make sure they don't accidentally allow someone to take someone else's account. Here is what we're going to do to fix this situation: Selali and his team are putting in extra hours over the weekends and try to bring the ticket count down. We have already hired twice as many CS staff to get ahead of this. We are looking for ways to optimize the CS training and tools to make our staff more efficient. Until I feel comfortable with our service level, either Selali or I will be posting these same stats every week on Friday evening before the weekend. We invite you to keep track, and hold us accountable. There should be no guess work. Hopefully we'll all watch these numbers go down and improve. I'm going to lock this topic, but feel free to discuss this in other threads. I just want a pinned area that we can add posts to it in order to keep the running tally. Thank you, Matt

Hi there, (My apologies for not posting this in Spanish) We appreciate all our players no matter what language they speak. Little Orbit has a history of translating their titles to many different languages. The situation with APB is a bit tricky. There are well over 600K words in the game. Typically translations cost $0.10 a word (or higher), so there is a fairly significant cost for each language we do. During the console port back in 2016, it appears that text changed significantly between both the PC and console titles. This caused huge chunks of text to show up in English regardless of the language you selected, because there weren't any new translations, so the decision was to remove many of the other languages. We are looking into how we can re-introduce localized text, but we need to figure out a long term solution for doing this that we can also maintain as new content gets added down the road. We have staffed many multi-lingual new GMs and Customer Support persons. We hope to support each community better in the future. As for your other questions, I don't honestly have those answers right now. Lastly for everyone else bringing negativity to this thread - give it a rest. There isn't an official Spanish forum, and the OP clearly stated this message was for LO. Thanks, Matt

Hi guys, Again, I think the OP has gotten enough answers, so I am going to lock this thead. But first, I'll answer this as definitively as I can. The Code of Conduct says: "Racial slurs, hate speech, threats, spam, and other forms of harassment will not be tolerated." The Nazi symbol and the confederate flag are two different issues: Displaying the Nazi symbol is illegal in some European countries where this game is sold. We are simply complying with the law when we find it used in game. The confederate flag is a question of intention. If you use it to recreate the General Lee from Dukes of Hazard - go for it. If you use it to harass another player, we will enforce the Code of Conduct. Personally I don't want to get caught up in legislating policy and making subjective decisions related to creative content. If something is reported by another player as offensive, then we're going to look at you're actions. Creating knowingly questionable content puts you at risk. If you want to take that risk, then you put the decision of your account's future in our hands and you should accept the consequences. Thanks, Matt

They are new district servers spread across existing worlds to help host more in-game players.

The new servers allow us to offload an instance here and there to help the servers perform better. But mostly we just need to keep pace with the growth.

Hi guys, The lag is a serious issue. We know it affects our players. I felt it was necessary to post my response here: Thanks, Matt

Hi guys, I have posted my response here. Thanks, Matt

Hi everyone, Several posts around the forums have mentioned network issues and especially bad lag over the weekend, so I thought I would address this publicly. First, I don't want to get in the habit of blaming "DDOS attacks" every time there is a problem. Sometimes that's the case. And sometimes it's not. It's important that G1 and LO be honest with the community when we mess up, and when our systems are being effected by external factors. We did get an especially heavy DDOS attack on Friday a few hours after the Q&A that took a bit to recover from. We switched on mitigation, and we were able to weather the storm. While we are on mitigation, our servers simply do not perform well. However, the attacked caused a side effect issue where some servers were "up" but disconnected from each other in a way that prevented new districts from spinning up and existing districts from responding. It took the better part of the weekend for us to both discover and resolve all of these issues. Simply put, the initial lag was out of our control, and we have been on mitigation ever since which creates lag, but we also messed up and made the situation worse over the weekend. Please accept my apologies. We're a little new a this. Second, we take this game and our players very seriously. An apology would be meaningless if we didn't describe how we are going to fix the issues. So here is how we're moving forward. We now have a new DDOS solution that is being tested, and if we find that the results are solid, then the new solution will all but eliminate the kind of lag caused by the existing mitigation solution. I'm already sick and tired of talking about how DDOS mitigation causes lag. I never want that to be an excuse for laggy servers again. We did some brainstorming, and we started on some new networking layer changes that will help with DDOS vulnerabilities. These sorts of changes need to be thoroughly tested, but we are committed to making this more difficult. We have provisioned several new servers. The player concurrency has doubled since we took over, and we are expanding our ability to host districts and provide a smoother in-game experience. We are altering our monitoring to not only determine whether servers are up, but also whether they care actively increasing their concurrent player count and other important metrics that let us know things are running correctly. Thanks, Matt

Hi everyone, You guys can use the Advanced Launcher for now, and more than likely you will be able to continue using it after the next patch. Thanks, Matt

Lol. No worries. You’re help is much appreciated.

Our intention is to build in support for what it does by the time the new engine upgrade is ready.

Hi there, Please accept my apology. We messed that up. The network was hit hard on Friday from a sizable DDOS attack, and we checked everything afterward to make sure the servers were all online. Unfortunately while the PS4 districts showed as online, they had lost their connection with each other, which prevented players from entering them. Sincerely, Matt

Hi there, This was a mistake on our part. We had a significant issue on Friday after a sizable DDOS attack. We checked all the servers to make sure they were online, but several of them had lost connectivity to each other. Sincerely, Matt

Hi there, I am sorry about that. It was an oversight on our part. We were informed some districts were down, and we fixed a couple others, but simply missed that one. You can read my full post at the bottom of page 2 about the issue: Thanks, Matt

Hi all, Good to know. We are attempting to merge the PC codebase onto the XBOX. Hopefully we can get that in for you guys. Thanks, Matt

Hi there, The consoles are on a completely different codebase than the PC. We are going to be staggering releases across both console and PC in order to combine both of those into a single codebase running on Unreal 3.5. So you will see updates, but not on the same schedule as the PC. Thanks, Matt

Hi everyone, I can assure you that there is no intention of removing any districts from any regions or platforms. We are new at this, and this was a mistake on our part - pure and simple. Here is an explanation of the issue: On Friday, we experienced the largest DDOS attack during the time we have owned APB. That knocked out access to the game for several minutes till we could get mitigation up. We corrected that, and did a round of spot checks to make sure everything was up. Unfortunately we failed to notice the several servers were up, but their connectivity to other servers was broken. We made more corrections, but it appears we still missed some console districts. I'm having the engineers go back and check connectivity on all the districts across all the regions / platforms to make sure everything is online. My apologies. Sincerely, Matt

Hi all, Just a note to say that we are working with the author of the Advanced Launcher. He has kindly made the code available, and we'll make sure there are no conflicts with BE. It isn't our intention to take away the tool. We just wanted to make sure it didn't interact negatively with the anti-cheat. Thanks, Matt

Hi everyone, I think this thread has had enough discussion about the OPs concept of turning video into Little Orbit to help ban cheaters. We are happy to look at video, and it some ways if we see enough of the same name, then we know to pay more attention, but video alone is difficult to use as evidence to ban. Viewing a player and seeing a "snap" is not necessarily an indication of cheating due to the way our camera works. Seeing as players have already deleted some of the content, I'm going to lock this thread. Thanks, Matt

Hi everyone, A lot of good points here. This is a complex, and I need to do more research on possible changes - if any. But I appreciate the feedback. Thanks, Mart