2 ms·
It doesn't matter how much testing you do – unfortunately some things can still fail. I'm much more interested in why the failure cascaded to other data centre
by matthewmacleod 9y ago
It doesn't matter how much testing you do – unfortunately some things can still fail.
I'm much more interested in why the failure cascaded to other data centres – that's exactly what shouldn't happen.
- hinkley 9y agoDidn’t AWS have a cascading failure last year because of a missing interlock on a script to take part of a data center offline? They discovered that it takes a lot longer than they remembered to reintroduce that many machines to the cluster. They came away from that incident with two or three action items.