"Overt catastrophic failure occurs when small, apparently innocuous failures join to create opportunity for a systemic accident. Each of these small failures is

"Overt catastrophic failure occurs when small, apparently innocuous failures join to create opportunity for a systemic accident. Each of these small failures is necessary to cause catastrophe but only the combination is sufficient to permit failure. ... complex systems run as broken systems. The system continues to function because it contains so many redundancies and because people can make it function, despite the presence of many flaws." https://how.complexsystems.fail/ https://how.complexsystems.fail/ This is as true for empires as it is for your distributed monolith on Kubernetes. A forgotten cup of coffee might be what actually causes a replication alert to be missed that brings everything to its knees, but if it was, the system was on the verge of failing, anyway. Robust systems can suffer many missed alerts without falling over. Put another way: for want of a nail, and for a systematic lack of maintenance of the other nails, the shoe was lost. For want of a horse, which could have been provided by the local stables which were somehow also not in operation, the rider was lost. For want of a message which should have been sent by multiple riders if it was that important, the battle was lost. No lessons on how to hold a kingdom together can be learned from a postmortem process that says, "Root cause: want of a nail."