5 ms·

My status page for an open source multiplayer game updates the status page for each server on a minute by minute bases. They can do better, they just don't wan

My status page for an open source multiplayer game updates the status page for each server on a minute by minute bases. They can do better, they just don't want to.

res0nat0r 5y ago
Large orgs (like Github) don't want or use automated status updates. There is usually always some service having issues in complex systems and immediately updating a public status page does more harm than good, including false alarms which may not affect anything public facing.
- Dylan16807 5y ago
  If you don't want to report sufficiently small issues, you can put that into the code, can't you? Besides that, how are you going to cause "more harm than good"?
  - derefr 5y ago
    More harm than good to the company's own long-term reputation. A status page is a kind of PR. Think of it like a policy for a flight attendant to come out into the cabin to tell everyone what's going on when the plane encounters turbulence. That policy is Public Relations -driven. You only do it if you expect that it's positive PR, compared to not doing it — i.e. if telling people what's going on is boosting your reputation compared to saying nothing at all. If a status page just makes your stakeholders think your service is crappy, such that you'd be better off with no status page at all... then why have a status page? It's not doing its job as a PR tool.
    - mjw1007 5y ago
      It seems to me you are now agreeing with the original « They can do better, they just don't want to. »
      - derefr 5y ago
        I read the line specifically as "The employees can do better; they just don't want to try." But that's not true. The company could do better. But the individual employees cannot. The individual employees are constrained by the profit motive of the company. They are not allowed by corporate policy to set up automatic status updates, for about the same reason they're not allowed to post their corporate log-in credentials: that the result would very likely be disastrous to the company's bottom line. (Though, really, the corporations in most verticals are in a race-to-the-bottom in most respects. Even if you treat GitHub as a single entity capable of coherent desires, it probably doesn't desire to avoid automatic status updates. It needs to avoid them, to survive in a competitive market where everyone else is also avoiding them. People — and corporations — do lots of things they don't want to do, to survive.)
derefr 5y ago
No, I don't think they can. An MMO is a very simple system, in that there's only one Service Level Indicator (SLI) that devs, shareholders, and players all agree on. That SLI is "can a player connect to the server, and perform regular gameplay actions, without a ridiculous amount of per-action latency." GitHub, meanwhile, is a million different things to a hundred million different people. Users of e.g. Homebrew, with its big monolithic ports system hosted as a github repo, have a very different SLI for Github than do users of some language-ecosystem package manager that allows you to pull deps directly from Github; than do people who depend on GitHub Actions to CI their builds on push; than do people doing code-review to others' PRs; than do people using Github mostly for its Wiki, or Issues, or downloading Releases, or Github Pages, or even just reading single-page-with-a-README repos, ala the various $FOO-awesome projects. For many of these use-cases, Github isn't degraded right now. For others, it is. If you ask for Github (or any service with this many different use-cases and stakeholders) to measure by the union of all these SLIs, then the service would literally never be not-degraded. In systems of sufficient scale, there's likely no point where every single component and feature and endpoint of the system is all working and robust and fast, all at once. Never has been, never will be. And anything less than just going for the union of all those SLIs, is asking Github to exercise human judgement over which kinds of service degradation qualify as part of their own SLOs. Which is exactly what they're doing. Certainly, internal to services like this, there are all sorts of alerting systems constantly going off to tell SREs what things need fixing. But not all of those things immediately, or even quickly, or even ever, translate to SLO violations. There are some outlier users whose use-cases just break the system's semantics, where those use-cases just aren't "in scope" for the SLO. As long as those users are only breaking the system for themselves, the degradation they experience won't ever translate to an SLO breakage.
- thih9 5y ago
  You seem to be applying different rules to MMOs and Github, and I don't understand why. I'd say that there are many ways of looking at this; there exist complex MMOs; and one could look at Github from the point of view of an average user. E.g., a bit tongue in cheek: > An MMO is a very simple system, in that there's only one Service Level Indicator (SLI) that devs, shareholders, and players all agree on. That SLI is "can a player connect to the server, and perform regular gameplay actions, without a ridiculous amount of per-action latency." Wouldn't you say that in an MMO of sufficient scale there's likely no point where every single component and feature and endpoint of the system is all working and robust and fast, all at once? > In systems of sufficient scale, there's likely no point where every single component and feature and endpoint of the system is all working and robust and fast, all at once. Never has been, never will be. Couldn't we redefine SLIs as "can the user connect to the server and perform regular user actions without a ridiculous amount of per-action latency"?

My status page for an open source multiplayer game updates the status page for each server on a minute by minute bases. They can do better, they just don't want to.

res0nat0r 5y ago

Large orgs (like Github) don't want or use automated status updates. There is usually always some service having issues in complex systems and immediately updating a public status page does more harm than good, including false alarms which may not affect anything public facing.

Dylan16807 5y ago
If you don't want to report sufficiently small issues, you can put that into the code, can't you? Besides that, how are you going to cause "more harm than good"?
- derefr 5y ago
  More harm than good to the company's own long-term reputation. A status page is a kind of PR. Think of it like a policy for a flight attendant to come out into the cabin to tell everyone what's going on when the plane encounters turbulence. That policy is Public Relations -driven. You only do it if you expect that it's positive PR, compared to not doing it — i.e. if telling people what's going on is boosting your reputation compared to saying nothing at all. If a status page just makes your stakeholders think your service is crappy, such that you'd be better off with no status page at all... then why have a status page? It's not doing its job as a PR tool.
  - mjw1007 5y ago
    It seems to me you are now agreeing with the original « They can do better, they just don't want to. »
    - derefr 5y ago
      I read the line specifically as "The employees can do better; they just don't want to try." But that's not true. The company could do better. But the individual employees cannot. The individual employees are constrained by the profit motive of the company. They are not allowed by corporate policy to set up automatic status updates, for about the same reason they're not allowed to post their corporate log-in credentials: that the result would very likely be disastrous to the company's bottom line. (Though, really, the corporations in most verticals are in a race-to-the-bottom in most respects. Even if you treat GitHub as a single entity capable of coherent desires, it probably doesn't desire to avoid automatic status updates. It needs to avoid them, to survive in a competitive market where everyone else is also avoiding them. People — and corporations — do lots of things they don't want to do, to survive.)

derefr 5y ago

No, I don't think they can. An MMO is a very simple system, in that there's only one Service Level Indicator (SLI) that devs, shareholders, and players all agree on. That SLI is "can a player connect to the server, and perform regular gameplay actions, without a ridiculous amount of per-action latency." GitHub, meanwhile, is a million different things to a hundred million different people. Users of e.g. Homebrew, with its big monolithic ports system hosted as a github repo, have a very different SLI for Github than do users of some language-ecosystem package manager that allows you to pull deps directly from Github; than do people who depend on GitHub Actions to CI their builds on push; than do people doing code-review to others' PRs; than do people using Github mostly for its Wiki, or Issues, or downloading Releases, or Github Pages, or even just reading single-page-with-a-README repos, ala the various $FOO-awesome projects. For many of these use-cases, Github isn't degraded right now. For others, it is. If you ask for Github (or any service with this many different use-cases and stakeholders) to measure by the union of all these SLIs, then the service would literally never be not-degraded. In systems of sufficient scale, there's likely no point where every single component and feature and endpoint of the system is all working and robust and fast, all at once. Never has been, never will be. And anything less than just going for the union of all those SLIs, is asking Github to exercise human judgement over which kinds of service degradation qualify as part of their own SLOs. Which is exactly what they're doing. Certainly, internal to services like this, there are all sorts of alerting systems constantly going off to tell SREs what things need fixing. But not all of those things immediately, or even quickly, or even ever, translate to SLO violations. There are some outlier users whose use-cases just break the system's semantics, where those use-cases just aren't "in scope" for the SLO. As long as those users are only breaking the system for themselves, the degradation they experience won't ever translate to an SLO breakage.

thih9 5y ago
You seem to be applying different rules to MMOs and Github, and I don't understand why. I'd say that there are many ways of looking at this; there exist complex MMOs; and one could look at Github from the point of view of an average user. E.g., a bit tongue in cheek: > An MMO is a very simple system, in that there's only one Service Level Indicator (SLI) that devs, shareholders, and players all agree on. That SLI is "can a player connect to the server, and perform regular gameplay actions, without a ridiculous amount of per-action latency." Wouldn't you say that in an MMO of sufficient scale there's likely no point where every single component and feature and endpoint of the system is all working and robust and fast, all at once? > In systems of sufficient scale, there's likely no point where every single component and feature and endpoint of the system is all working and robust and fast, all at once. Never has been, never will be. Couldn't we redefine SLIs as "can the user connect to the server and perform regular user actions without a ridiculous amount of per-action latency"?