Load problems don't go away when the load disappears. If the system isn't engineered very carefully (this takes a lot of work!), you may have cascading failures

Load problems don't go away when the load disappears. If the system isn't engineered very carefully (this takes a lot of work!), you may have cascading failures that may take hours to resolve, especially if you have bad retry policies (their mention of thundering herd problem seems to indicate that they might). We wrote a bit about this here: https://landing.google.com/sre/sre-book/chapters/addressing-cascading-failures/ https://landing.google.com/sre/sre-book/chapters/addressing-... I would strongly caution anyone who thinks this subject is trivial, just add a bit of load shedding and you're done. I wrote a bit about my team's work (including a simplified view of some of the considerations that go into how we do retries) here: https://landing.google.com/sre/sre-book/chapters/handling-overload/ https://landing.google.com/sre/sre-book/chapters/handling-ov...