4 ms·
This seems to miss a couple of fairly likely scenarios. 1) us-east-1 goes down, and a whole bunch of important AWS backplane goes down with it, so it's impossi
by bigiain 5y ago
This seems to miss a couple of fairly likely scenarios.
1) us-east-1 goes down, and a whole bunch of important AWS backplane goes down with it, so it's impossible to bring everything back up again in other regions.
2) us-east-1 goes down, and the thundering herd stomps all over the other regions making it impossible to use them.
3) either scenario 1 or 2 happen, and another thundering herd of people who've made plans to stand everything up in Azure/GCP take those down as well.
- sofixa 5y agoAs far as we know publicly, AWS regions are very separated and even if us-east-1 is entirely dead, the impact on the others should be minimal and mostly cosmetic ( i hope they've fixed the status page that was hosted in S3 us-east-1).
- palijer 5y agoWell, at the end of the day in this situation it is still humans that thought up the idea to separate the regions. I presume it is also humans who want to keep us-east-1 alive. Neither of those conditions (use1 going down or the cross DC contamination) are intentional. If us-east-1 goes dead because of a condition that humans failed to anticipate, then it stands to reason that the other regions are actually dependent on use1 in a way humans didn't anticipate.
- bigiain 5y agoI’ve had problems in ap-southeast-2 not being able to provision resources when us-east-1 is having problems. Stuff that’s running works fine. Turning on new stuff, not so much. From memory auto scaling has failed me at least once in that situation too… or maybe that was spot fleets not rebalancing?
- jnsaff2 5y agoThe understanding is that the services which you use in "global" region are hosted in us-east-1 .. CloudFront, Route 53, IAM. So outages in us-east-1 can cripple a lot more than you'd think. Your EC2 proably still runs but I would not count on AutoScaling or being able to log in and do admin tasks. I'm sure AWS has also thought about it and the good part about us-east-1 is that it has 5 or more zones so it's harder to take it down.
- lozenge 5y agoCertain API endpoints are not considered part of the availability guarantees. You might be able to verify and create IAM tokens, but not create new IAM roles, new S3 buckets etc. AWS sometimes advises that these operations shouldn't be done automatically, but not everybody listens.
- jmnicolas 5y agoAs I understand in this case a scenario is a reason why us-east-1 goes down, not the consequences.