2 ms·
A good postmortem... and a clear example of a Type 1 Outage. There are pretty much only two kinds of outage in a decently-run system: 1. Two or three things go
by snewman 12y ago
A good postmortem... and a clear example of a Type 1 Outage. There are pretty much only two kinds of outage in a decently-run system:
1. Two or three things go wrong at once. Some of the problems are spontaneous, and some were always broken but it took the other problem(s) to uncover it. (Here, the SSD failure was presumably spontaneous, but the "not completely successful" routing engine failover sounds like a case of rarely-used-hence-poorly-tested.)
2. A systemic failure hits all redundant components at once (DDOS, fat-fingered global configuration change, calendar bug, etc.)