15 ms·
How I Almost Destroyed a £50M War Plane and the Normalisation of Deviance (2016)
- Xylakant 7y agoAbout 2 weeks ago a German Luftwaffe Global 5000 Businessjet missed the runway and nearly crashed in an emergency landing. The currently suspected cause: Faulty maintenance and incomplete preflight checks. http://www.airliners.de/in-schoenefeld-jet-flugbereitschaft/49799 http://www.airliners.de/in-schoenefeld-jet-flugbereitschaft/...
- voltagex_ 7y agoPodcast seems to be 404 on libsyn. I was worried it'd gone Spotify exclusive but the correct feed URL is https://feed.pippa.io/public/shows/5bb78b332630bb48399f39fe https://feed.pippa.io/public/shows/5bb78b332630bb48399f39fe and the episode URL is https://feed.pippa.io/public/streams/5bb78b332630bb48399f39fe/episodes/5bb78b4fac8922f843285be0.mp3 https://feed.pippa.io/public/streams/5bb78b332630bb48399f39f... (as an alternative to the linked article).
- coldcode 7y agoI see this sort of complacent deviance all the time at my job. Thankfully we don't can't kill people with it. But insular organizations seem to thrive on doing things "their way" thinking it right, when in fact to anyone else it would be total lunacy. If people saw what we did I think people on HN would have a good laugh.
- tomohawk 7y ago> Before you try and change the world just have a look at the foundation from which you are starting it from. So true. People who think they're doing the right thing can cause so much damage.
- DoofusOfDeath 7y agoIMO the article is excellently written and very compelling. I suggest people read it for enjoyment, despite the dry title.
- MatthewWilkes 7y agoWhat do you do on a daily basis that makes "How I Almost Destroyed a £50 million War Plane" sound dry?
- DoofusOfDeath 7y agoGood point. I was focusing on the "... Normalization of Deviance" part when I wrote that.
- jacquesm 7y agoIt's really nice to have a term to pin to a vague concept that had been floating around in my head for at least a decade. I've come across instances of this repeatedly but never knew quite how to describe it properly to others. It's a treacherous slope where the first step is usually completely un-noticed and by the time people do notice it is way too late. We've always done it like this is the oft heard excuse but on closer examination that usually isn't true. Typically at some point in the past there was a much tighter regime and the present day situation would have definitely not passed muster. But by inching towards it with very small steps critical thresholds can be exceeded without notice. Thanks for posting this. It will help me professionally to formulate my opinions on such situations more clearly and will serve as a reference point for future discussion.
- JorgeGT 7y agoI would say this is the engineering incarnation of the Broken window theory of the early eighties, if you want a term which is much more known: https://en.wikipedia.org/wiki/Broken_windows_theory https://en.wikipedia.org/wiki/Broken_windows_theory
- totoglazer 7y agoFWIW, that’s a very politically loaded term. I would recommend avoiding it unless you intend to bait people into an unrelated conversation. I also think it’s not a very accurate analogy.
- myself248 7y agoWhat's inaccurate about it? My understanding is "Things looking bad allows people to assume that worse stuff will be tolerated". The way it was enforced seems to be where the political baggage comes in, but the fundamental theory shows up all over the place. If not identical, it at least seems closely adjacent. I'm curious to hear your perspective; what's wrong with the analogy?
- cobbzilla 7y agoIt’s unfortunate that “broken window theory” has become synonymous with the awful and racist police tactics introduced in NYC. Broken window theory is real: imagine a random person in spotlessly clean city versus a city where there is trash everywhere. More people would litter in the latter city, I don’t think that is too controversial. Thus a run-down neighborhood with lots of broken windows will normalize behaviors that would be stigmatized elsewhere. The unfortunate part was when NY adopted “broken windows policing”, which is to go into those neighborhoods and make lots of arrests for petty crimes, as if replacing all the broken windows might somehow reduce crime. Of course this is pretty dumb, since entrenched normalization of deviance (decades of bad/discriminatory policy) can’t simply be fixed by zealous application of punitive measures.
- empath75 7y agoClimate change is another example of this probably. Except the whole planet instead of a plane.
- intertextuality 7y agoA lot of incidents seem to arise from normalization of deviance + self regulation (especially in the moment). And hubris. The author's situation reminds me of the Tenerife airport disaster- absolutely macabre and sobering. [0] [0]: https://en.wikipedia.org/wiki/Tenerife_airport_disaster https://en.wikipedia.org/wiki/Tenerife_airport_disaster
- gchamonlive 7y agoWhen it was being described the checklist a pilot has to go through before taking off, it resembled, though quite crudely and without life or death implications, the lengthy checklist the developers at our company had to follow back when we deployed everything manually. We had synchronization steps to pull modifications, couple compilation steps, we had to check if configuration files were added in order to accommodate extra services, there was cache mechanisms that had to be cleared, after that we needed to check for database migrations, then restart the systems, run the little tests we had and manually check if nothing broke. It happened often to see code conflicts, 40 to 50 with varying difficulty every release. It was insane and never quite worked, we always got weary of release days and never wanted to go through them as we didn't trust the system (with good reason). Automation and good practices brought back confidence, so I was wondering two things. First most obvious, how come do pilots trust a system that has these layers of manual security checks with their lives. And second, can't it all be automated? I understand aircrafts must have independent modules and every level of integration implies extra risk, but at least in security critical tests, automation would come a great way in ensuring crew safety, wouldn't it? Is it too hard? Systems are not flexible to implement automatic security checklists or is it just a culture in the field to check it all manually?
- ohazi 7y ago> First most obvious, how come do pilots trust a system that has these layers of manual security checks with their lives. And second, can't it all be automated? It's one thing to have a script check the return code of a test suite. Often times aircraft checklists include things like physically verifying that something is present and moving in the correct manner when commanded to do so. Obviously you can add sensors, but then how do you check that the sensor is working? At the end of the day, most of the manual checklist items are still for a human to look at or listen to a thing and make a judgement call.
- sokoloff 7y agoMany aircraft do have configuration warning systems, but they are not exhaustive. And, as a sibling commenter mentioned, many of them are physical inspection items. (Fuselage free of ice and snow, fuel appropriate for the mission, appropriate fuel loaded, pitot cover removed, etc)
- ramshanker 7y agoThis was a really good reminder of engineering wisdom. Over the year I find myself and my team (civil-structural engineers) doing even more and more riskier designs. Things we wouldn't have considered OK during our early years. We do get review and approvals from seniors, but maybe their acceptance is also "normalisation of deviance" as described in the article. Sharing it with my team.
- Zenst 7y agoWould it be a case of the norm was to over-engineer (thinking victorian times mostly as examples of structures standing and still in use) compared with today in which we have a finer insight into the maths, which allows us to engineer just-enough. Or is it a case of market/management pressure pushing the limits that is the driver? With those pushing the limits getting a competitive edge - that draws others into competing in the same way.
- richardhod 7y agoCan you give (, perhaps anonymised) examples? This makes me wonder which of the new breeds of skyscraper are actually a lot less safe than perhaps older ones. I have read the same with cruise ships as we have seen with a disaster or two on those
- btrettel 7y agoWhen I think of "normalization of deviance" I focus more on mundane things like speeding. I'm a cyclist. People often ask me why cyclists break the law so frequently. The studies I've looked at don't suggest cyclists break the law any more frequently than drivers do, but they do break different laws. (To my knowledge I follow every applicable traffic law.) When I ask the driver I'm speaking with if they speed I frequently get a justification of speeding as if it's perfectly safe because "everyone does it". In terms of problems people should focus on, cyclists tend to break laws in ways that mostly harm only themselves (e.g., running a red light), while drivers break laws in ways that tend to harm others in addition to themselves (e.g., speeding, failure to yield, etc.). The fact that drivers vastly outnumber cyclists also makes the priorities even more clear.
- GordonS 7y ago> cyclists tend to break laws in ways that mostly harm only themselves (e.g., running a red light) Running a red light could easily harm others - a driver could swerve to avoid you, crashing in the process; if a driver hits you because you ran a light, that could also cause a chain of accidents; and if a driver killed you because you ran a red light, that could seriously harm their wellbeing.
- computerfriend 7y agoThey said "mostly" and, while your comment is undoubtedly true, it is a little bit disingenuous to suggest those things as likely when cyclists are the most vulnerable road users.
- tomc1985 7y ago> disingenuous to suggest those things as likely when cyclists are the most vulnerable road users. Why is it disingenuous? Cyclists can be jerks too
- nitrogen 7y agoAs a pedestriab I've been nearly hit by a cyclist way more often than nearly hit by a car. Running red lights on a bike in a city is very dangerous to people crossing the street.
- RangerScience 7y agoThis reminds me a lot of this idea: https://www.lesswrong.com/posts/Kbm6QnJv9dgWsPHQP/schelling-fences-on-slippery-slopes https://www.lesswrong.com/posts/Kbm6QnJv9dgWsPHQP/schelling-... "Normalization of deviance" would be the mechanism by which a slope becomes slippery; a schelling fence (in this case... a checklist?) is then a possible fix, even if it's just to trigger a re-evaluation of "normal"
- vermontdevil 7y agoThink of the global financial crisis of 2008 when the economy collapsed because the banks hadn’t been properly regulated because they convinced the authorities that they could do it themselves. Also Boeing and the MCAS fiasco
- blunte 7y agoI'm amazed that outside a wartime situation that flying a plane like that with a known equipment failure (of that nature especially) was allowed. And if it wasn't allowed, I'm surprised it didn't end up in a grounding of the pilot.
- viraptor 7y agoIt wasn't just the pilot. It was also a few people who heard the idea and said that's ok. You can't fix systemic issues by punishing selected few. Aviation seems to be pretty good at that.
- fuzzy2 7y agoNice read. I often notice something similar during software development. It's simple stuff like "I have to do a full rebuild or it won't work" or "I have to build twice" etc pp. I cannot understand how others can simply ignore what could indicate a fundamental problem. It's fucking terrifying.
- dboreham 7y agoI notice this too. However I've come to see that the building twice folks are cranking on writing code and getting praise from management while I'm finding some bug in yarn and not getting any actual work done..
- gav 7y agoAn interesting paper about systems on the edge of failure is "'Going solid': a model of system dynamics and consequences for patient safety" by Cook and Rasmussen: https://qualitysafety.bmj.com/content/14/2/130 https://qualitysafety.bmj.com/content/14/2/130 There's a good overview in this talk: "Architectural Patterns of Resilient Distributed Systems" by Ines Sombra https://youtu.be/ohvPnJYUW1E?list=PL1Fqq0rxeNehyA4eWbuCwizkqLeL9-6aE&t=589 https://youtu.be/ohvPnJYUW1E?list=PL1Fqq0rxeNehyA4eWbuCwizkq...
- vmh1928 7y agoThe concept of Normalization of Deviance was mentioned in this thread in March about some comments made on Twitter by a software engineer about the 737 Max crashes. https://news.ycombinator.com/item?id=19414775 https://news.ycombinator.com/item?id=19414775
- calabin 7y agoTried to read your article, but the way the email prompt loaded made it impossible to dismiss so I entered a false address. I'm running the latest version of Chrome on Ubuntu 16.04 - hope that helps you diagnose this issue. Here's a screenshot: https://imgur.com/kwXdIsa https://imgur.com/kwXdIsa
- burfog 7y agoWhat I find odd is that he chose to get his 0g flight by flying a parabola, in the style of the vomit comet, instead of just flying in a roll. For example, roll by 200 degrees and then raise the nose by about 20 degrees. (probably can't just roll 180 degrees without stalling)
- eridius 7y ago"Normalization of Deviance" sounds like the name of a ship from the Culture series.
- pacaro 7y agoIn my experience this also applies to dating, it’s really easy to overlook things in the early stages of a relationship (or what might become a relationship) that turn out to be quite significant later.
- chriselles 7y agoComplacency creep. Talking to a few Royal Australian Air Force folks last year, apparently they had a similar-ish culture of “she’ll be right” arrogance/confidence, and then they had a cluster of accidents that caused a foundation level rethink and rebuild of a safety/process culture into every flight and every high risk activity. It sounds like accident #’s have collapsed since then. Be careful, aviation fanboy writing, not a qualified pilot/tech/engineer.
- spsrich 7y agoAnybody reminded of Captain Flashheart while reading this?
- dsfyu404ed 7y agoEveryone here seems to be caught up in waxing poetic about how the normalization of evidence is a bad thing and talking about preventing it but there's another side to that coin. If everybody is breaking the process 9x/10 it's not "normalization of deviance", it's because you've got a half baked process that sucks and the people on the ground know it and work around it. Obviously you could fill a library with all the material written on how to write organizational policy but it's not as black and white as "violating policy = bad"
- dang 7y agoA few past related discussions: https://hn.algolia.com/?query=Normalization%20of%20Deviance%20points%3E20&sort=byDate&dateRange=all&type=story&storyText=false&prefix=false&page=0 https://hn.algolia.com/?query=Normalization%20of%20Deviance%...
- nan0 7y agoThis read reminded me of one of the famous TOP GUN movie quotes: "A good pilot is compelled to always evaluate what's happened, so he can apply what he's learned. Up there, we gotta push it."
- bond 7y agoTop Gun...
- awinter-py 7y agoFirst heard this term in the danluu 'wtf wtf' blogpost -- so useful. Also 'fast jet performance' is such a cool podcast topic.
- kristianp 7y agoThis one: https://danluu.com/wat/ https://danluu.com/wat/ Great articles there.
- toss1 7y agoExcellent read and a key concept. I'd had a general idea about this, but it was crystalized by a vehicle dynamics engineer and racer who mentored me in both engineering and race driving. He said: "You need to know the difference between skill and getting away with something" I.e., just because you survive an event does not mean that it is reliable. If it is based on skill, you can replicate it and reliably survive. If it based on luck, well . . . take the bit of grace you were given by the Gods and learn your lesson to not do it again. Mostly, THINK.
- DoctorOetker 7y ago"How I Almost Destroyed a £50M War Plane and the Normalisation of Deviance" versus "How I Almost Destroyed a £50M War Plane [while nearly killing myself and my colleague] and the Normalisation of Deviance"
- lmm 7y agoYeah I thought that was interesting. Perhaps an artifact of how soldiers are trained to regard their own lives versus their equipment?