3 ms·
Good to see an analysis emphasizing the metastable failure mode in EC2, rather than getting bogged down by the DNS/Dynamo issue. The Dynamo issue, from their ti
by tptacek 1y ago
Good to see an analysis emphasizing the metastable failure mode in EC2, rather than getting bogged down by the DNS/Dynamo issue. The Dynamo issue, from their timeline, looks like it got fixed relatively quickly, unlike EC2, which needed a fairly elaborate SCRAM and recovery process that took many hours to execute.
A faster, better-tested "restart all the droplet managers from a known reasonable state" process is probably more important than finding all the Dynamo race conditions.
- thundergolfer 1y agoI was motivated by your back-and-forth in the original AWS summary to go and write this post :)
- tptacek 1y agoIt's good, and I love that you brought the Google SRE stuff into it.