5 ms·

No, I don't think they can. An MMO is a very simple system, in that there's only one Service Level Indicator (SLI) that devs, shareholders, and players all agre

by derefr 5y ago

No, I don't think they can. An MMO is a very simple system, in that there's only one Service Level Indicator (SLI) that devs, shareholders, and players all agree on. That SLI is "can a player connect to the server, and perform regular gameplay actions, without a ridiculous amount of per-action latency." GitHub, meanwhile, is a million different things to a hundred million different people. Users of e.g. Homebrew, with its big monolithic ports system hosted as a github repo, have a very different SLI for Github than do users of some language-ecosystem package manager that allows you to pull deps directly from Github; than do people who depend on GitHub Actions to CI their builds on push; than do people doing code-review to others' PRs; than do people using Github mostly for its Wiki, or Issues, or downloading Releases, or Github Pages, or even just reading single-page-with-a-README repos, ala the various $FOO-awesome projects. For many of these use-cases, Github isn't degraded right now. For others, it is. If you ask for Github (or any service with this many different use-cases and stakeholders) to measure by the union of all these SLIs, then the service would literally never be not-degraded. In systems of sufficient scale, there's likely no point where every single component and feature and endpoint of the system is all working and robust and fast, all at once. Never has been, never will be. And anything less than just going for the union of all those SLIs, is asking Github to exercise human judgement over which kinds of service degradation qualify as part of their own SLOs. Which is exactly what they're doing. Certainly, internal to services like this, there are all sorts of alerting systems constantly going off to tell SREs what things need fixing. But not all of those things immediately, or even quickly, or even ever, translate to SLO violations. There are some outlier users whose use-cases just break the system's semantics, where those use-cases just aren't "in scope" for the SLO. As long as those users are only breaking the system for themselves, the degradation they experience won't ever translate to an SLO breakage.

thih9 5y ago
You seem to be applying different rules to MMOs and Github, and I don't understand why. I'd say that there are many ways of looking at this; there exist complex MMOs; and one could look at Github from the point of view of an average user. E.g., a bit tongue in cheek: > An MMO is a very simple system, in that there's only one Service Level Indicator (SLI) that devs, shareholders, and players all agree on. That SLI is "can a player connect to the server, and perform regular gameplay actions, without a ridiculous amount of per-action latency." Wouldn't you say that in an MMO of sufficient scale there's likely no point where every single component and feature and endpoint of the system is all working and robust and fast, all at once? > In systems of sufficient scale, there's likely no point where every single component and feature and endpoint of the system is all working and robust and fast, all at once. Never has been, never will be. Couldn't we redefine SLIs as "can the user connect to the server and perform regular user actions without a ridiculous amount of per-action latency"?
- derefr 5y ago
  > and one could look at Github from the point of view of an average user. My point was that Github has no "average user." Github is like Microsoft Word: each user only uses 10% of the features, but it's a different 10% for every user. Yes, there are some features that are in the critical path for all users (loading the toplevel repo view in the website might be one); but for any given particular user, there will be plenty of other features that are also in their critical path. An MMO, meanwhile, does have an "average user"; in fact, MMOs have ideal users. An MMO's goal is to induce every user (player) to play the game a certain way, so that the company can concentrate their resources on making that particular play experience as polished as possible. There is, per se, an idiomatic "rut" in the road that players can "click into", ending up doing exactly the same short-term game loops that every other player before and after them has also done when playing the game. MMOs can be reduced to a single SLO: can the ideal player have fun playing the game at the moment? GitHub cannot be reduced to a single SLO, because there is no ideal user of GitHub. There are probably two or three thousand separate "ideal users" (= critical, non-universal user stories) for GitHub. > Wouldn't you say that in an MMO of sufficient scale there's likely no point where every single component and feature and endpoint of the system is all working and robust and fast, all at once? No, not really; MMOs have a complexity ceiling by operational necessity. They aren't composed of a ridiculous sprawling array of components. They might use Service-Oriented Architecture, but in the end, you don't scale an MMO vertically by making more and more complex clustered systems with master-to-master replication and so forth. You scale MMOs by either pure-vertical hardware scaling, or by horizontal shared-nothing sharding. (The key thing to realize about MMO servers is that they're OLTP servers — they need to track a whole bunch of users doing a whole bunch of simple actions at once; and therefore they can't really be doing overly-much computation on those actions, lest they lose shared-realtime verisimilitude.)
  - thih9 5y ago
    I think you underestimate the complexity of MMOs. They can host truly massive events, e.g. https://en.m.wikipedia.org/wiki/Battle_of_B-R5RB https://en.m.wikipedia.org/wiki/Battle_of_B-R5RB . With so many different play styles and optional components (guilds, pvp, official forums, paid content, user made content, etc), I’d say defining an ideal MMO gamer is just as easy as defining an ideal SaaS user. Not sure if any MMO reaches Github level, most likely not. But I don’t think there is a ceiling or any sort of hard distinction; i.e. I think 5 years from now we could have a MMO with complexity of today’s Github. Maybe it will be called a metaverse though.

by derefr 5y ago

thih9 5y ago

You seem to be applying different rules to MMOs and Github, and I don't understand why. I'd say that there are many ways of looking at this; there exist complex MMOs; and one could look at Github from the point of view of an average user. E.g., a bit tongue in cheek: > An MMO is a very simple system, in that there's only one Service Level Indicator (SLI) that devs, shareholders, and players all agree on. That SLI is "can a player connect to the server, and perform regular gameplay actions, without a ridiculous amount of per-action latency." Wouldn't you say that in an MMO of sufficient scale there's likely no point where every single component and feature and endpoint of the system is all working and robust and fast, all at once? > In systems of sufficient scale, there's likely no point where every single component and feature and endpoint of the system is all working and robust and fast, all at once. Never has been, never will be. Couldn't we redefine SLIs as "can the user connect to the server and perform regular user actions without a ridiculous amount of per-action latency"?

derefr 5y ago
> and one could look at Github from the point of view of an average user. My point was that Github has no "average user." Github is like Microsoft Word: each user only uses 10% of the features, but it's a different 10% for every user. Yes, there are some features that are in the critical path for all users (loading the toplevel repo view in the website might be one); but for any given particular user, there will be plenty of other features that are also in their critical path. An MMO, meanwhile, does have an "average user"; in fact, MMOs have ideal users. An MMO's goal is to induce every user (player) to play the game a certain way, so that the company can concentrate their resources on making that particular play experience as polished as possible. There is, per se, an idiomatic "rut" in the road that players can "click into", ending up doing exactly the same short-term game loops that every other player before and after them has also done when playing the game. MMOs can be reduced to a single SLO: can the ideal player have fun playing the game at the moment? GitHub cannot be reduced to a single SLO, because there is no ideal user of GitHub. There are probably two or three thousand separate "ideal users" (= critical, non-universal user stories) for GitHub. > Wouldn't you say that in an MMO of sufficient scale there's likely no point where every single component and feature and endpoint of the system is all working and robust and fast, all at once? No, not really; MMOs have a complexity ceiling by operational necessity. They aren't composed of a ridiculous sprawling array of components. They might use Service-Oriented Architecture, but in the end, you don't scale an MMO vertically by making more and more complex clustered systems with master-to-master replication and so forth. You scale MMOs by either pure-vertical hardware scaling, or by horizontal shared-nothing sharding. (The key thing to realize about MMO servers is that they're OLTP servers — they need to track a whole bunch of users doing a whole bunch of simple actions at once; and therefore they can't really be doing overly-much computation on those actions, lest they lose shared-realtime verisimilitude.)
- thih9 5y ago
  I think you underestimate the complexity of MMOs. They can host truly massive events, e.g. https://en.m.wikipedia.org/wiki/Battle_of_B-R5RB https://en.m.wikipedia.org/wiki/Battle_of_B-R5RB . With so many different play styles and optional components (guilds, pvp, official forums, paid content, user made content, etc), I’d say defining an ideal MMO gamer is just as easy as defining an ideal SaaS user. Not sure if any MMO reaches Github level, most likely not. But I don’t think there is a ceiling or any sort of hard distinction; i.e. I think 5 years from now we could have a MMO with complexity of today’s Github. Maybe it will be called a metaverse though.