> one person was celebrated because they were the most knowledgeable at handling every kind of on-call situation. In hindsight, this was a red flag. It is a re

> one person was celebrated because they were the most knowledgeable at handling every kind of on-call situation. In hindsight, this was a red flag. It is a red flag indeed, but “in hindsight”? From my experience as an engineering manager, in addition to what article mentions, the following things are useful: * having some kind of error codes (e.g. PRJNAME-E0032) so that search through documentation or knowledge base is unambiguous and instantaneous * Any incident that was not handled by support/ops routinely, according to known procedures and was escalated to someone on call, gets an agenda item on a regular root cause analysis meeting. The result must be someone’s issue report: engineers on my team, infrastructure team, vendor, downstream or upstream application team etc. * regular check-in meeting with support/ops where the central agenda item should be going over statistics about escalations for certain period and even incidents handled routinely (e.g. it was low disk space due to excessive logging and was handled without involving developers - do we log too much, do we need bigger disks etc?)