16 ms·
There might be good advice in here for an average day, but as a piece of timely advice targeted to this particular outage it's very, very annoying -- fingernail
by mechanical_fish 16y ago
There might be good advice in here for an average day, but as a piece of timely advice targeted to this particular outage it's very, very annoying -- fingernails-on-a-chalkboard annoying, to those of us in the trenches -- because it is full of crap. The premise is faulty, and half of these strategies are useless today, because:
A) We've seen downtime in multiple zones, not just one. The big problem seems like it might be localized to one zone, but there are spillover effects. See below.
B) We have extensive EBS snapshots, thank you very much, and, yes, they span availability zones. Unfortunately, they can't be reliably reinstantiated as volumes in any zone in US-East.
If one is going to chase ambulances one could at least provide some advice that is actually topical. Like, e.g., an article on the tradeoffs and design choices needed to run a service that spans regions or even companies, not just zones.
Sorry to be so testy, but it's not a good day for platitudes.
- webmonkeyuk 16y agoNo need for apologies. I must admit that it's very much an idiot's guide aimed at smaller setups or people less familar with web operations. I can assure you there's no copy/paste and I'm surprised at the interest it's had.
- mechanical_fish 16y agoDon't worry; hopefully by tomorrow this disaster will be over and we'll be back to the state where your advice is really good advice that everyone needs to hear and I'll be sipping delicious drinks somewhere.
- robotadam 16y agoIn addition, some ELBs are seeing failures with routing, so that wouldn't help, either.