10 ms·

There's an old(ish) episode of the Stack Overflow podcast where they interview a couple folks from the site reliability engineering team. I think this is the on

There's an old(ish) episode of the Stack Overflow podcast where they interview a couple folks from the site reliability engineering team. I think this is the one: https://stackoverflow.blog/2017/06/12/podcast-111-sre-occasionally-maintaining-infrastructure-hate/ https://stackoverflow.blog/2017/06/12/podcast-111-sre-occasi... Somewhere in there, one of the SREs said something that's stuck with me. Roughly, he said that the concept of human error is useless. Every time a human error escalates into a real problem, the deeper problem is a design flaw. At scale, human errors are a certainty, and, while documentation and training is far from useless, no amount of it can change that fact. It's therefore a critical responsibility of systems designers to design their systems to be resilient to mistakes.

mybandisbetter 6y ago
This is also a big concept in the medical world. There's an influential report on this from the 90s called "To Err is Human." https://pubmed.ncbi.nlm.nih.gov/25077248/ https://pubmed.ncbi.nlm.nih.gov/25077248/
capableweb 6y ago
This is also not a new idea that even comes from the software world. I'm not entirely sure of the original of such ideas, but the first time I heard about was in conjunction with airplanes and their UX. Basically, crashes and other issues from flying is never attributed to the operator, but the UX of the controls as if it was done better, would have prevented the operator of committing the error in the first place.
- CPLX 6y ago
  Indeed. One of the really core things you learn when you dig deeper into aviation is that essentially every major disaster is a chained combination of multiple extremely unlikely failures in sequence. Which is a testament to both the incredible success this approach has had, as well as the impossibility of perfection.
  - Gare 6y ago
    So called "Swiss cheese model": https://en.wikipedia.org/wiki/Swiss_cheese_model https://en.wikipedia.org/wiki/Swiss_cheese_model
  - alpaca128 6y ago
    I once saw a detailed breakdown of a well-known Concorde crash and it's staggering how many coincidences came together to mess up those people's day. Iirc a spare part for another plane was faulty, fell off onto the runway, the Concorde's wheels hit it and exploded due to the strain, one of the rubber chunks hit the fuel tanks causing a leak and another shredded some live wires, leading to ignition of the leaking fuel.
    - andi999 6y ago
      Isn't almost everything the fuel tank? So I would say these events are not all independent.
    - GuB-42 6y ago
      It wasn't the first time the Concorde had exploded tires. There already was several "close calls". The last one was for real, with the power of hindsight, they should have addressed the problem long before the deadly crash. For me, the best illustration is the Tenerife airport disaster, the deadliest accident in the history of aviation. Two Boeing 747 crashing into each other on the runway during takeoff. Circumstances included fog, problems in the control tower, nonstandard phraseology, radio interference and overworked personnel.
- EForEndeavour 6y ago
  Where do you draw the line between blaming the UX and blaming the training process, or the inevitable cases where an irreversible decision made under uncertainty turned out to be the wrong one? Some level of training is certainly needed, so in my naive view anyway, it makes no sense to attribute every error to UX. For example, let's say a pilot belly-lands because they failed to lower their landing gear, possibly because they were distracted by some other equipment failure, or possibly because sometimes qualified and experienced pilots do this. This error could have been avoided with, say, an automated spoken "gear up" warning instead of a nonverbal warning horn (I have no idea if some aircraft already have this). But it could also have been avoided by longer and more extensive training, right?
  - capableweb 6y ago
    > But it could also have been avoided by longer and more extensive training, right? Sure, it maybe could, but why risk it? Humans are also humans, and even pilots with endless amount of flight time does mistakes sometimes, because we're all humans. The idea is to work with the idea that sometimes we fail, even if we are well trained, so when failures happen, it's easy to recover. > Where do you draw the line between blaming the UX and blaming the training process, or the inevitable cases where an irreversible decision made under uncertainty turned out to be the wrong one? Basically, it's always the UX. If there is a case where the pilot made the wrong decision, something needs to change so if a pilot with similar experience, wouldn't have made that decision with a different UX. Basically force the pilots to do the right decision, always. I'm not 100% on the history of all of this, so if someone is more familiar with it, please correct me. But I think it started with "Crew resource management" to ensure proper training of the crew. It has gone through many iterations where less and less blame is being made on the crew and more on the equipment. I think the point of the current iteration basically boils down to "Human-error will happen, let's plan for it and if human-error lead to a plane going down, let's fix it so the same error would be prevented".
    - NovemberWhiskey 6y ago
      >Basically, it's always the UX. If there is a case where the pilot made the wrong decision, something needs to change so if a pilot with similar experience, wouldn't have made that decision with a different UX. Basically force the pilots to do the right decision, always. That's not realistic. If you think it is realistic, please take a look at the investigation report for PK8303. Some pilots will disregard alerts, override safeties and ignore procedures: there's only so much a system can do to promote good decision-making. https://www.caapakistan.com.pk/Upload/SIBReports/AAIB-431.pdf https://www.caapakistan.com.pk/Upload/SIBReports/AAIB-431.pd...
Jtsummers 6y ago
That's a fundamental thesis of systems safety work. "Human error" is a convenient, but fairly useless, designation for the cause behind something. Why was the human able to perform the erroneous act? If the person went out of their way to perform the act, for instance they manually turned off all alarms because they wanted to take a nap, then sure, it's (largely) human error. But then you can still ask, why were they even able to turn off critical alarms? Why was there a single person involved in this apparently crucial position? What allowed us to hire someone like Homer Simpson into the nuclear power plant in the first place? And do we really want to rely on eeny-meeny-miny-moe to solve our problems? Maybe training needs to be improved, including evaluation of training results through simulations.
- speeder 6y ago
  I think the only cases of major system failures that could be blamed on human error, are ones where people INTENTIONALLY did wrong stuff, sadly, these cases do exist and were very lethal. 1. First nuclear accident: the theory is one operator was upset that another operator was screwing his wife, and removed the control rods 100% when he was scheduled to work on them, causing a sudden spike in temperature and then a water hammer that killed everyone in the building. (and made it JUMP!!!) 2. Chernobyl, although many people (specially anti-nuclear) blame the accident on the design, it was obvious deliberate error, even if the intentions weren't malicious, the operators ignored alarms, then DISABLED the alarms, and intentionally pushed the reactor past safety limits, even if it didn't had the design flaw it had, it was possible they would still find a way to blow it up if they kept their behaviour. 3. Depressed airline pilot, locked co-pilot out of the cockpit and crashed on the ground on purpose. There are probably other situations out there... thing is, those are situations no matter how well the system is designed, it will still happen, it is impossible to make a "malicious-proof" system, after all, even a rock, with zero moving parts, can be used in the "wrong way" and kill someone.
  - MauranKilom 6y ago
    More info for 1.: https://en.wikipedia.org/wiki/SL-1 https://en.wikipedia.org/wiki/SL-1
lmkg 6y ago
This is also the approach taking in aviation safety, and similar safety-critical systems. The pilot is treated as part of the system. Just like any other component, it's a part that can fail, and its error states can be modeled. Performance and reliability can be affected by training and certification, but just like anything else you're shooting for a threshold of tolerance for errors, not a complete removal of errors.
- jack_riminton 6y ago
  Parts are also designed on the basis that they WILL fail at some point so the whole system can still perform WHEN this happens
  - toomuchtodo 6y ago
    Great example: Cirrus has a new personal jet, the Vision Jet. It has a feature called Safe Return Emergency Land [1]. If the pilot is incapacitated, a passenger can press the Big Red Button to activate the system, at which point the aircraft switches to autonomous mode to select the closest airport to navigate to and auto land at. Even the pilot can fail and the system will attempt to recover from this failure mode. [1] https://cirrusaircraft.com/story/introducing-safe-return-emergency-autoland/ https://cirrusaircraft.com/story/introducing-safe-return-eme...
    - skynetv2 6y ago
      Amazing tech! Thanks for sharing.
    - bluGill 6y ago
      Pretty much all new commercial jets have an autoland mode. If something happens to the pilots get into the cockpit (this is the hard part!) fast put on a headset, find the push to talk button and yell help. You need to be fast enough that the radio frequency to use hasn't changed (this is almost as hard as getting in!) as the plane moved on, but if you get a response whoever it is will get in contact with control for you and from there on they will tell you what buttons to push. Of course if you are in an old airplane (private plane, or legally retired but still in use in some poor country) all bets are off.
professoretc 6y ago
I took a design course in grad school and one of the things they emphasized was that it's impossible for everyone to be wrong about something. If everyone is using something the wrong way, then its the design that's wrong, not the people.
- base698 6y ago
  https://en.wikipedia.org/wiki/The_Design_of_Everyday_Things https://en.wikipedia.org/wiki/The_Design_of_Everyday_Things "Doors can have two states: open or closed. They should not require instruction."
  - egypturnash 6y ago
    I’m looking at the front door of my apartment as I type this, and I can see sixteen states for it quite easily. It’s got a lock on the doorknob, a deadbolt, and a big window with a blind. All of these can be open or closed independently of each other; add on the basic “is it open or closed” and that’s four binary choices for 16 states. Also there is that fun state of “open but will lock behind you because the doorknob lock does not prevent you from closing the door, hope you have your keys on you or someone inside” though that’s maybe more a special case in the state transition table than an actual state?
    - base698 6y ago
      Do those things have affordances or do you pull out your state table?
    - sbierwagen 6y ago
      Another fun state: open the door, then turn the deadbolt. Now the door can't be closed! Attempt to close it hard enough and you can bend the deadbolt shaft. Now the deadbolt can't be retracted, and the door is entirely disabled! An arcane edge case, but something a clumsy child could do in less than a minute. How could we tolerate such an awful design? Who is responsible for this catastrophe?
  - wombatpm 6y ago
rational_indian 6y ago
> At scale, human errors are a certainty, while documentation and training is far from useless, no amount of it can change that fact. It's therefore a critical responsibility of systems designers to design their systems to be resilient to mistakes. If only the people who insist on using unsafe languages understood this. Their position is that you should get better at not making mistakes.
- missblit 6y ago
  I work with C++ on a daily basis and the position is actually closer to "sandbox anything tricky, and run everything through fuzzers and sanitizers." Anyone who thinks they can overcome 100% of memory-unsafety bugs through looking at the code real hard is fooling themselves. But I'm not convinced there's a lot of people who actually think that (maybe there's some people who just don't care or never considered it)
  - rational_indian 6y ago
    This has not been my experience. Most people don't bother with these things. Memory safety / undefined behavior is a non issue for them.
Tempest1981 6y ago
Ok, but... there are some powertools that cannot be made completely safe. Like git.
JustSomeNobody 6y ago
So, they should have made the Challenger completely incapable of launching in cold wx, for example?
- SamBam 6y ago
  That certainly seems like a reasonable thing to me, now that we know that to be an issue. Why wouldn't you design a system that prevents it from taking off in conditions that are known to cause a crash? Obviously they may not know these conditions ahead of time. But preventing user error for known problems seems obvious.
- kleer001 6y ago
  I bet that was on a list of considerations and was dismissed.
- wtallis 6y ago
  That sounds like you're trying to propose a solution that's as far as possible from addressing the root cause and any "human error". NASA didn't need a shuttle that would refuse to take off in cold weather. They needed engineering assessments that realistically quantified and clearly communicated the risks of launching in cold weather. Given accurate information about the capabilities of the shuttle, mission control would have waited for warmer weather or implemented countermeasures.
  - eidedxueq 6y ago
    The Challenger explosion was entirely political. It was the result of the structure of human relationships and not any engineering discipline. Had the social structure been politically different on that day they would have acted on the available engineering information by refusing to launch. It was the political environment that goaded them into taking more risk than they had previously decided was acceptable.
  - JustSomeNobody 6y ago
    The Challenger explosion was human error. It wasn't a design flaw, it was human error. That was my point.
natchy 6y ago
That concept is from the book “the design of everyday things” by Dan Norman. A Seminal book in UX psychology.
hinkley 6y ago
Given enough eggs, a chef will eventually throw the eggs in the trash and put the shells in the pan. Given enough orders, someone will do that every single day.
jasonwatkinspdx 6y ago
This book is a great short manifesto on exactly that point: https://www.amazon.com/Field-Guide-Understanding-Human-Error/dp/1472439058 https://www.amazon.com/Field-Guide-Understanding-Human-Error... It's written by someone that does airliner crash investigations. His central point is that "human error" as a term functions to redirect blame away from the people who establish systems and procedures. It blames the last domino vs the people who stacked them. It's a quick breezy read, and you'll get the main points within the first 30 min or so of reading. I've found it useful for getting these ideas across to people though, especially more generic business types where "no blame post mortem" strikes them as some care bear nonsense rather than being an absolutely essential tool to reduce future incidents.
kthejoker2 6y ago
I forget which book exactly, but one of Scott Adams's has a story from a reader about a process consultant who comes from a background in aviation into some typical Fortune 500 company and tells them the trick to improvement is just to never make a mistake, ever. And then proceeds to use reductio ad absurdum to demonstrate that if you can be perfect for 5 seconds, you can string those together to be perfect for 5 minutes ... 5 hours ... 5 days .. your whole life! And when someone in the crowd pushed back on this idea, the consultant said something like, "If the airlines took that same approach to flying, we'd have 1,000 accidents a year!" And then maybe the reader looked it up later and it turned out there were about 1,000 flying accidents a year.

mybandisbetter 6y ago

This is also a big concept in the medical world. There's an influential report on this from the 90s called "To Err is Human." https://pubmed.ncbi.nlm.nih.gov/25077248/ https://pubmed.ncbi.nlm.nih.gov/25077248/

capableweb 6y ago

This is also not a new idea that even comes from the software world. I'm not entirely sure of the original of such ideas, but the first time I heard about was in conjunction with airplanes and their UX. Basically, crashes and other issues from flying is never attributed to the operator, but the UX of the controls as if it was done better, would have prevented the operator of committing the error in the first place.

CPLX 6y ago
Indeed. One of the really core things you learn when you dig deeper into aviation is that essentially every major disaster is a chained combination of multiple extremely unlikely failures in sequence. Which is a testament to both the incredible success this approach has had, as well as the impossibility of perfection.
- Gare 6y ago
  So called "Swiss cheese model": https://en.wikipedia.org/wiki/Swiss_cheese_model https://en.wikipedia.org/wiki/Swiss_cheese_model
- alpaca128 6y ago
  I once saw a detailed breakdown of a well-known Concorde crash and it's staggering how many coincidences came together to mess up those people's day. Iirc a spare part for another plane was faulty, fell off onto the runway, the Concorde's wheels hit it and exploded due to the strain, one of the rubber chunks hit the fuel tanks causing a leak and another shredded some live wires, leading to ignition of the leaking fuel.
  - andi999 6y ago
    Isn't almost everything the fuel tank? So I would say these events are not all independent.
  - GuB-42 6y ago
    It wasn't the first time the Concorde had exploded tires. There already was several "close calls". The last one was for real, with the power of hindsight, they should have addressed the problem long before the deadly crash. For me, the best illustration is the Tenerife airport disaster, the deadliest accident in the history of aviation. Two Boeing 747 crashing into each other on the runway during takeoff. Circumstances included fog, problems in the control tower, nonstandard phraseology, radio interference and overworked personnel.
EForEndeavour 6y ago
Where do you draw the line between blaming the UX and blaming the training process, or the inevitable cases where an irreversible decision made under uncertainty turned out to be the wrong one? Some level of training is certainly needed, so in my naive view anyway, it makes no sense to attribute every error to UX. For example, let's say a pilot belly-lands because they failed to lower their landing gear, possibly because they were distracted by some other equipment failure, or possibly because sometimes qualified and experienced pilots do this. This error could have been avoided with, say, an automated spoken "gear up" warning instead of a nonverbal warning horn (I have no idea if some aircraft already have this). But it could also have been avoided by longer and more extensive training, right?
- capableweb 6y ago
  > But it could also have been avoided by longer and more extensive training, right? Sure, it maybe could, but why risk it? Humans are also humans, and even pilots with endless amount of flight time does mistakes sometimes, because we're all humans. The idea is to work with the idea that sometimes we fail, even if we are well trained, so when failures happen, it's easy to recover. > Where do you draw the line between blaming the UX and blaming the training process, or the inevitable cases where an irreversible decision made under uncertainty turned out to be the wrong one? Basically, it's always the UX. If there is a case where the pilot made the wrong decision, something needs to change so if a pilot with similar experience, wouldn't have made that decision with a different UX. Basically force the pilots to do the right decision, always. I'm not 100% on the history of all of this, so if someone is more familiar with it, please correct me. But I think it started with "Crew resource management" to ensure proper training of the crew. It has gone through many iterations where less and less blame is being made on the crew and more on the equipment. I think the point of the current iteration basically boils down to "Human-error will happen, let's plan for it and if human-error lead to a plane going down, let's fix it so the same error would be prevented".
  - NovemberWhiskey 6y ago
    >Basically, it's always the UX. If there is a case where the pilot made the wrong decision, something needs to change so if a pilot with similar experience, wouldn't have made that decision with a different UX. Basically force the pilots to do the right decision, always. That's not realistic. If you think it is realistic, please take a look at the investigation report for PK8303. Some pilots will disregard alerts, override safeties and ignore procedures: there's only so much a system can do to promote good decision-making. https://www.caapakistan.com.pk/Upload/SIBReports/AAIB-431.pdf https://www.caapakistan.com.pk/Upload/SIBReports/AAIB-431.pd...

Jtsummers 6y ago

That's a fundamental thesis of systems safety work. "Human error" is a convenient, but fairly useless, designation for the cause behind something. Why was the human able to perform the erroneous act? If the person went out of their way to perform the act, for instance they manually turned off all alarms because they wanted to take a nap, then sure, it's (largely) human error. But then you can still ask, why were they even able to turn off critical alarms? Why was there a single person involved in this apparently crucial position? What allowed us to hire someone like Homer Simpson into the nuclear power plant in the first place? And do we really want to rely on eeny-meeny-miny-moe to solve our problems? Maybe training needs to be improved, including evaluation of training results through simulations.

speeder 6y ago
I think the only cases of major system failures that could be blamed on human error, are ones where people INTENTIONALLY did wrong stuff, sadly, these cases do exist and were very lethal. 1. First nuclear accident: the theory is one operator was upset that another operator was screwing his wife, and removed the control rods 100% when he was scheduled to work on them, causing a sudden spike in temperature and then a water hammer that killed everyone in the building. (and made it JUMP!!!) 2. Chernobyl, although many people (specially anti-nuclear) blame the accident on the design, it was obvious deliberate error, even if the intentions weren't malicious, the operators ignored alarms, then DISABLED the alarms, and intentionally pushed the reactor past safety limits, even if it didn't had the design flaw it had, it was possible they would still find a way to blow it up if they kept their behaviour. 3. Depressed airline pilot, locked co-pilot out of the cockpit and crashed on the ground on purpose. There are probably other situations out there... thing is, those are situations no matter how well the system is designed, it will still happen, it is impossible to make a "malicious-proof" system, after all, even a rock, with zero moving parts, can be used in the "wrong way" and kill someone.
- MauranKilom 6y ago
  More info for 1.: https://en.wikipedia.org/wiki/SL-1 https://en.wikipedia.org/wiki/SL-1

lmkg 6y ago

This is also the approach taking in aviation safety, and similar safety-critical systems. The pilot is treated as part of the system. Just like any other component, it's a part that can fail, and its error states can be modeled. Performance and reliability can be affected by training and certification, but just like anything else you're shooting for a threshold of tolerance for errors, not a complete removal of errors.

jack_riminton 6y ago
Parts are also designed on the basis that they WILL fail at some point so the whole system can still perform WHEN this happens
- toomuchtodo 6y ago
  Great example: Cirrus has a new personal jet, the Vision Jet. It has a feature called Safe Return Emergency Land [1]. If the pilot is incapacitated, a passenger can press the Big Red Button to activate the system, at which point the aircraft switches to autonomous mode to select the closest airport to navigate to and auto land at. Even the pilot can fail and the system will attempt to recover from this failure mode. [1] https://cirrusaircraft.com/story/introducing-safe-return-emergency-autoland/ https://cirrusaircraft.com/story/introducing-safe-return-eme...
  - skynetv2 6y ago
    Amazing tech! Thanks for sharing.
  - bluGill 6y ago
    Pretty much all new commercial jets have an autoland mode. If something happens to the pilots get into the cockpit (this is the hard part!) fast put on a headset, find the push to talk button and yell help. You need to be fast enough that the radio frequency to use hasn't changed (this is almost as hard as getting in!) as the plane moved on, but if you get a response whoever it is will get in contact with control for you and from there on they will tell you what buttons to push. Of course if you are in an old airplane (private plane, or legally retired but still in use in some poor country) all bets are off.

professoretc 6y ago

I took a design course in grad school and one of the things they emphasized was that it's impossible for everyone to be wrong about something. If everyone is using something the wrong way, then its the design that's wrong, not the people.

base698 6y ago
https://en.wikipedia.org/wiki/The_Design_of_Everyday_Things https://en.wikipedia.org/wiki/The_Design_of_Everyday_Things "Doors can have two states: open or closed. They should not require instruction."
- egypturnash 6y ago
  I’m looking at the front door of my apartment as I type this, and I can see sixteen states for it quite easily. It’s got a lock on the doorknob, a deadbolt, and a big window with a blind. All of these can be open or closed independently of each other; add on the basic “is it open or closed” and that’s four binary choices for 16 states. Also there is that fun state of “open but will lock behind you because the doorknob lock does not prevent you from closing the door, hope you have your keys on you or someone inside” though that’s maybe more a special case in the state transition table than an actual state?
  - base698 6y ago
    Do those things have affordances or do you pull out your state table?
  - sbierwagen 6y ago
    Another fun state: open the door, then turn the deadbolt. Now the door can't be closed! Attempt to close it hard enough and you can bend the deadbolt shaft. Now the deadbolt can't be retracted, and the door is entirely disabled! An arcane edge case, but something a clumsy child could do in less than a minute. How could we tolerate such an awful design? Who is responsible for this catastrophe?
- wombatpm 6y ago

rational_indian 6y ago

> At scale, human errors are a certainty, while documentation and training is far from useless, no amount of it can change that fact. It's therefore a critical responsibility of systems designers to design their systems to be resilient to mistakes. If only the people who insist on using unsafe languages understood this. Their position is that you should get better at not making mistakes.

missblit 6y ago
I work with C++ on a daily basis and the position is actually closer to "sandbox anything tricky, and run everything through fuzzers and sanitizers." Anyone who thinks they can overcome 100% of memory-unsafety bugs through looking at the code real hard is fooling themselves. But I'm not convinced there's a lot of people who actually think that (maybe there's some people who just don't care or never considered it)
- rational_indian 6y ago
  This has not been my experience. Most people don't bother with these things. Memory safety / undefined behavior is a non issue for them.

Tempest1981 6y ago

Ok, but... there are some powertools that cannot be made completely safe. Like git.

JustSomeNobody 6y ago

So, they should have made the Challenger completely incapable of launching in cold wx, for example?

SamBam 6y ago
That certainly seems like a reasonable thing to me, now that we know that to be an issue. Why wouldn't you design a system that prevents it from taking off in conditions that are known to cause a crash? Obviously they may not know these conditions ahead of time. But preventing user error for known problems seems obvious.
kleer001 6y ago
I bet that was on a list of considerations and was dismissed.
wtallis 6y ago
That sounds like you're trying to propose a solution that's as far as possible from addressing the root cause and any "human error". NASA didn't need a shuttle that would refuse to take off in cold weather. They needed engineering assessments that realistically quantified and clearly communicated the risks of launching in cold weather. Given accurate information about the capabilities of the shuttle, mission control would have waited for warmer weather or implemented countermeasures.
- eidedxueq 6y ago
  The Challenger explosion was entirely political. It was the result of the structure of human relationships and not any engineering discipline. Had the social structure been politically different on that day they would have acted on the available engineering information by refusing to launch. It was the political environment that goaded them into taking more risk than they had previously decided was acceptable.
- JustSomeNobody 6y ago
  The Challenger explosion was human error. It wasn't a design flaw, it was human error. That was my point.

natchy 6y ago

That concept is from the book “the design of everyday things” by Dan Norman. A Seminal book in UX psychology.

hinkley 6y ago

Given enough eggs, a chef will eventually throw the eggs in the trash and put the shells in the pan. Given enough orders, someone will do that every single day.

jasonwatkinspdx 6y ago

This book is a great short manifesto on exactly that point: https://www.amazon.com/Field-Guide-Understanding-Human-Error/dp/1472439058 https://www.amazon.com/Field-Guide-Understanding-Human-Error... It's written by someone that does airliner crash investigations. His central point is that "human error" as a term functions to redirect blame away from the people who establish systems and procedures. It blames the last domino vs the people who stacked them. It's a quick breezy read, and you'll get the main points within the first 30 min or so of reading. I've found it useful for getting these ideas across to people though, especially more generic business types where "no blame post mortem" strikes them as some care bear nonsense rather than being an absolutely essential tool to reduce future incidents.

kthejoker2 6y ago

I forget which book exactly, but one of Scott Adams's has a story from a reader about a process consultant who comes from a background in aviation into some typical Fortune 500 company and tells them the trick to improvement is just to never make a mistake, ever. And then proceeds to use reductio ad absurdum to demonstrate that if you can be perfect for 5 seconds, you can string those together to be perfect for 5 minutes ... 5 hours ... 5 days .. your whole life! And when someone in the crowd pushed back on this idea, the consultant said something like, "If the airlines took that same approach to flying, we'd have 1,000 accidents a year!" And then maybe the reader looked it up later and it turned out there were about 1,000 flying accidents a year.