Jump to content
MattScott

Network outage 2018-07-29 7am PDT

Recommended Posts

Hi everyone,

This morning at 7am PDT one of our circuits had rolling connectivity issues which caused servers to be unavailable. That circuit is back up, but the engineers are looking at the core issue. When these types of network issues happen, we are forced to do a round of server reboots to make sure Districts and World servers can talk to each other properly. 

We've spent quite a bit of time patching server performance and looking at forward-facing network systems, and I believe we've made real progress there. However clearly there are still more cobwebs to clean. Personally it's frustrating to see failures in these "back of house" systems that cause downtime on a weekend like this. Tomorrow I'm going to sit with the networking team to start evaluating them one by one to see if we can reduce the possibility of this kind of downtime in the future.

Thanks,
Matt

  • Like 9
  • Thanks 37
  • Dislike 2

Share this post


Link to post
Share on other sites
2 minutes ago, MattScott said:

However clearly there are still more cobwebs to clean. Personally it's frustrating to see failures in these "back of house" systems that cause downtime on a weekend like this.

Sounds like technical debt due to the previous management not keeping up with software development well enough at all

Appreciate the update though, thanks
  • Thanks 1

Share this post


Link to post
Share on other sites

communicating the 'what's and why's' to the community is a solid customer service experience.

thanks to you and your team for the weekend service.

it does not go unappreciated!

Share this post


Link to post
Share on other sites
1 minute ago, uraputz said:

communicating the 'what's and why's' to the community is a solid customer service experience.

thanks to you and your team for the weekend service.

it does not go unappreciated!

+1, despite the annoyance of servers going down, atleast we know whats going on. Big ups to Matt and LO
  • Like 1

Share this post


Link to post
Share on other sites
10 minutes ago, MattScott said:

Hi everyone,

This morning at 7am PDT one of our circuits had rolling connectivity issues which caused servers to be unavailable. That circuit is back up, but the engineers are looking at the core issue. When these types of network issues happen, we are forced to do a round of server reboots to make sure Districts and World servers can talk to each other properly. 

We've spent quite a bit of time patching server performance and looking at forward-facing network systems, and I believe we've made real progress there. However clearly there are still more cobwebs to clean. Personally it's frustrating to see failures in these "back of house" systems that cause downtime on a weekend like this. Tomorrow I'm going to sit with the networking team to start evaluating them one by one to see if we can reduce the possibility of this kind of downtime in the future.

Thanks,
Matt

whoever downvoted is an pleasant fellow. G1C wouldn't give us this they wouldn't care. Matt does.
  • Like 1

Share this post


Link to post
Share on other sites
Just now, tries said:

How long is downtime going to be?

The downtime should be just long enough to reboot servers. The network is back up now.
  • Like 2
  • Thanks 1
  • Haha 2

Share this post


Link to post
Share on other sites
3 minutes ago, MattScott said:
6 minutes ago, tries said:

How long is downtime going to be?

The downtime should be just long enough to reboot servers. The network is back up now.
Would this possibly have an impact on what was suppose to be announced this week of new patches and new content ???  

Share this post


Link to post
Share on other sites

nothing but obstacles you can over come! thank you for your patience and work matt, much love for you

Edited by scrim
error

Share this post


Link to post
Share on other sites

Thank you Matt for communicating with us. Even if it seems trivial, it means a lot to us.

Share this post


Link to post
Share on other sites

agradeço a equipe pelo desempenho e dedicação que tem com todos os clientes ... parabéns
 

amigos preciso de ajuda , so consigo entrar no distrito financial! quando tento entrar em waterfront ou outro distrito fight club , nao entra!
fica carregando por horas....
so tou jogando em financial, preciso upar minha conta oque devo fazer pra concertar isso?
 

Edited by Alencar Tattoo
  • Dislike 1

Share this post


Link to post
Share on other sites

updoot 

Share this post


Link to post
Share on other sites
8 minutes ago, Alencar Tattoo said:

com todos os clientes e parabéns
 

amigos preciso de ajuda, assim quando tento entrar em (walter front) ou outro distrito (fight club), nao entra!
fica carregando por horas ....
so tou jogando em (financial), preciso upar minha conta oque devo fazer pra concertar isso?
 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...