5 ms·

Those five 9s don't come easy. Sometimes you have to prop them up :)

jjoonathan 5y ago
I wonder how often outages really happen. The official page is nonsense, of course, and we only collectively notice when the outage is big enough that lots of us are affected. On AWS, I see about a 3:1 ratio of "bump in the night" outages (quickly resolved, little corroboration) to mega too-big-to-hide outages. Does that mirror others' experiences?
- Spivak 5y ago
  If you count any time AWS is having a problem that impacts our production workloads then I think it's about 5:1. Dealing with "AWS is down" outages are easy because I can just sit back and grab some popcorn, it's the "dammit I know this is AWS's fault" outages that are a PITA because you count yourself lucky to even get a report in your personalized dashboard.
  - jjoonathan 5y ago
    Yep. Random aside: any chance you are related to the Calculus on Manifolds Spivak?
    - Spivak 5y ago
      Nope, just a fan. It was the book that pioneered my love of math.
    - clh1126 5y ago
      I had to log in to say, that one of my favorite quotes of all time I found in Calculus on Manifolds. He says that any good theorem is worth generalizing, and I've generalized that to any life rule.
kylemh 5y ago
https://aws.amazon.com/compute/sla/ https://aws.amazon.com/compute/sla/ looks like only four 9's
- dotancohen 5y ago
  > looks like only four 9's That's why the Germans are such good engineers. Did the drives fail? Nein. Did the CPU overheat? Nein. Did the power get cut? Nein. Did the network go down? Nein. That's "four neins" right there.
1-6 5y ago
It’s hard to measure what five-9 is because you have to wait around until a 0.00001 occurs. Incentivizing post-mortems are absolutely critical in this case.
- notinty 5y ago
  It's 0.001; the first 2 9's count. 5N = 99.999% 3N = 99.9% 1N5 = 95% 5N is <43m12s downtime per month.
  - 1-6 5y ago
    I considered writing it as a percent but then decided against using it and moving the decimal instead. But good info for clarification.
JoelMcCracken 5y ago
Every time someone asks to update the status page, managers say "nein"

I wonder how often outages really happen. The official page is nonsense, of course, and we only collectively notice when the outage is big enough that lots of us are affected. On AWS, I see about a 3:1 ratio of "bump in the night" outages (quickly resolved, little corroboration) to mega too-big-to-hide outages. Does that mirror others' experiences?

Spivak 5y ago
If you count any time AWS is having a problem that impacts our production workloads then I think it's about 5:1. Dealing with "AWS is down" outages are easy because I can just sit back and grab some popcorn, it's the "dammit I know this is AWS's fault" outages that are a PITA because you count yourself lucky to even get a report in your personalized dashboard.
- jjoonathan 5y ago
  Yep. Random aside: any chance you are related to the Calculus on Manifolds Spivak?
  - Spivak 5y ago
    Nope, just a fan. It was the book that pioneered my love of math.
  - clh1126 5y ago
    I had to log in to say, that one of my favorite quotes of all time I found in Calculus on Manifolds. He says that any good theorem is worth generalizing, and I've generalized that to any life rule.

kylemh 5y ago

https://aws.amazon.com/compute/sla/ https://aws.amazon.com/compute/sla/ looks like only four 9's

dotancohen 5y ago
> looks like only four 9's That's why the Germans are such good engineers. Did the drives fail? Nein. Did the CPU overheat? Nein. Did the power get cut? Nein. Did the network go down? Nein. That's "four neins" right there.

1-6 5y ago

It’s hard to measure what five-9 is because you have to wait around until a 0.00001 occurs. Incentivizing post-mortems are absolutely critical in this case.

notinty 5y ago
It's 0.001; the first 2 9's count. 5N = 99.999% 3N = 99.9% 1N5 = 95% 5N is <43m12s downtime per month.
- 1-6 5y ago
  I considered writing it as a percent but then decided against using it and moving the decimal instead. But good info for clarification.

JoelMcCracken 5y ago

Every time someone asks to update the status page, managers say "nein"