SREs I know talk about aviation incidents all the time, spinning it into a “what lessons do we learn?” - Aviation incidents are investigated and the investigat

SREs I know talk about aviation incidents all the time, spinning it into a “what lessons do we learn?” - Aviation incidents are investigated and the investigation is typically bent on discovering root causes, not assigning blame. Do the same thing with bugs & security breaches, you’ll learn more. - Pilots spend time in simulators practicing unusual situations like water landings or engine failure—do the same thing in software, simulate failure scenarios and intrusion events and see how your engineers and technicians handle those events. You develop new training materials and redesign systems based on what you learned in simulations. - Pilots rely heavily on checklists rather than memory to ensure that all steps have been completed. This translates well to QA, deployment, various tasks people you need to do in production like migrations or whatnot. - Pilots require good UX to do their job and cannot be overloaded with too many tasks. Sometime an incident will be traced to dangerous things that don’t seem dangerous when you look at the UI (like thrust reversers), or traced to systems that assigned too many duties to pilots (who get overloaded). For example, if you have a button that deletes a database, it should require a serious confirmation step, like a molly guard on a plane. - Tasks must be explicitly delegated to individual people (which is what happens on an airplane). For example, in the Gimli Glider incident, the flight engineer role was eliminated and the engineer's responsibilities were not explicitly delegated to the pilot or copilot, but assigned to “both”. Same should be true for tasks in software engineering and incident response. We also talk about how medicine seems uninterested in learning these lessons, for various reasons.