Disclaimer: I work at AWS, but not in any of the groups that build and maintain any of the infrastructure components. My team is on the B2B applications side of

Disclaimer: I work at AWS, but not in any of the groups that build and maintain any of the infrastructure components. My team is on the B2B applications side of the house. We can be just as affected by an AWS infrastructure outage as anyone. However, we also have the possibility of seeing some of the things that are going on internally to fix the problems. So, we can at least feel better about not being able to do our work for the day. I can tell you this — being multi-region is a lot harder to get right than anyone gives it credit for. Even just plain multi-AZ is harder than most people give it credit for. There’s a lot of stuff going on under the hood. And there’s all sorts of unintentional dependencies between systems that you didn’t realize until things broke just the right way. Trying to do multi-cloud? I wouldn’t wish that on my worst enemy. The way cloud-scale works, there’s always something weird going on somewhere. Always. A well-architected system will try to be rugged and resilient to those failures, but there’s always going to be a limit to the kinds of failure modes you can predict and prepare for. We saw that when I was the Sr. Internet Mail Administrator at AOL from 95-97, we see this today in our group at AWS, and my friends at Google have reported the same.