4 ms·
>> The data are clear that humans are really bad at taking the time to do things that are well understood to incontrovertibly reduce the risk of rare but catast
by YeGoblynQueenne 9y ago
>> The data are clear that humans are really bad at taking the time to do
things that are well understood to incontrovertibly reduce the risk of rare
but catastrophic events.
The big problem with the "normalisation of deviance" is that humans are both
very smart and also get bored, or irriated, very easily. The example of the
anesthesiologist who turned off the anoxia warning is indicative.
But if that's how humans work, then that's something that needs to be taken
into account when designing, essentially, security systems. We can't ignore
the propensity of people to look for shortcuts to repetitive or boring tasks
any more than we can ignore the fact that we tend to have our hands at the top
of our body, or that we can't withstand extremes of temperature.
We wouldn't design a machine that had all its controls placed near the ground,
too low to be manipulated easily by the average human; we wouldn't send
unprotected humans to work in environments where temperatures could reach 1000
°C. Why would we give people tasks that would bore them to death or make them
so mad that they would risk harm to themselves or others to avoid performing
them?
The (IAEA) [1] has a bunch of papers on radiological accidents on its website and
they are full of situations where people have literally hacked through
security measures to stop them from blasting themselves with lethal doses of
radiation- and fried themselves to death as a result.
For instance, see [2], [3] and [4]. These are three cases, one in El Salvador,
one in Israel and one in Belarus, where an experienced operator of an
irradiation facility overrode security mechanisms to enter the irradiation
chamber and unblock a mechanism transporting packages over a radioactive
source [5]. In [3], the chamber was designed like a D&D dungeon, complete with
a concrete maze to absorb radiation, and actual traps: motion and pressure
sensors that, when triggered, would immediately place the source at a safe
position inside a dry pit and even a retractable section of floor near the
maze entrance that created a pit too long to be jumped over. They never found
out exactly how the operator crossed that pit. But- he did and paid for his
ingenuity with his life.
We are smart little monkeys, we get bored easily and we always find a way to
cause untold carnage. Those are factors that no design can ever afford to
ignore.
__________________
[1] International Atomic Energy Agency.
[2] http://www-pub.iaea.org/books/IAEABooks/3798/The-Radiological-Accident-in-Soreq http://www-pub.iaea.org/books/IAEABooks/3798/The-Radiologica...
[3] http://www-pub.iaea.org/MTCD/publications/PDF/Pub847_web.pdf http://www-pub.iaea.org/MTCD/publications/PDF/Pub847_web.pdf
[4] http://www-pub.iaea.org/books/IAEABooks/4712/The-Radiological-Accident-at-the-Irradiation-Facility-in-Nesvizh http://www-pub.iaea.org/books/IAEABooks/4712/The-Radiologica...
[5] To sterilise the contents.