8 ms·
The Reliability Trap: The crash of Emirates flight 521 (2020)
- roxgib 4y agoWhat seems to link a lot of these cases is that the pilots tell the plane to do something and the plane decides it knows better. Throw up all the warning you want, but a computer shouldn't be able to override the pilot's decisions.
- kayodelycaon 4y agoSome overrides make sense. Airbus’s flight envelope protection prevents pilots from stalling the plane. A great many accidents have happened due to stalls. Airplanes have never been safer, despite many more planes flying. Crashes of airliners is rare. If anything, that points to automation greatly improving safety. Thing of it this way, many more pilots accidentally hit the TOGA button on the ground. The automation has surely prevented more accidents than it has caused.
- sofixa 4y ago> Airbus’s flight envelope protection prevents pilots from stalling the plane. To be fair, that's what Boeing's MCAS was also trying to do. It's just that Airbus aren't incompetent and/or criminally negligent, and don't hide the existence of such automations.
- kayodelycaon 4y agoMCAS wasn’t intended as a safety system. It was designed to make the Max behave like previous 737 models.
- sofixa 4y agoIt was designed to avoid stalling in conditions under which the previous 737 wouldn't, and thus to make the Max behave the same.
- kayodelycaon 4y agoThat doesn’t make it a safety system. It wasn’t integrated as part of any kind of flight envelope protection and it wasn’t necessary to fly a plane of that design safely. The issue was they decided to use automation to avoid training pilots on how the plane actually behaves. That kills people.
- unsafecast 4y agoBut we can have both. Make the button yell at you if the circumstances make it dangerous to use it. The fact that they didn't know is a failure of the system, which should've let them know immediately that the button didn't actually do the thing.
- kayodelycaon 4y agoI agree but I also wonder if there are good reasons more alarms aren’t used. Far worse than the occasional silent failure are too many false alarms. The medical profession has this problem. If you get too many alerts, people ignore them. There was a series of incidents where pilots were routinely pulling circuit breakers to silence takeoff config warnings going off while taxiing. Eventually a plane took off without flaps and a lot of people were killed. Automation is very complex and the solutions aren’t always so straight forward. People forget there is a serious risk of adding an alarm causing more accidents. Should there be an alarm here? Seems obvious there should be and there better be a damned good reason there isn’t.
- deleted 4y ago[deleted]
- unsafecast 4y agoThat's a fair point. I guess attacking the problem from another angle helps then - simplifying the computer UI and minimising the number of situations where an alarm could be warranted in the first place. But yes, this stuff is very complex and I'm (we're?) only judging this from the outside.
- dmitrygr 4y agoThere are many accidents in aviation that show that 99% reliable automation kills. It is just reliable enough that you don’t expect it to fail, and you are not ready to handle if it does. This is why self driving cars of today, that claim that humans should take over in case of failure are nonsense. We have a huge body of knowledge showing that when faced with a system that almost always works, humans suck at overseeing it and taking over when needed.
- throwawaylinux 4y agoThis case was perfectly reliable automation. The automated systems behaved exactly as designed. The point of the article isn't that the automation just needs to be made more reliable and that will solve the problem, it's that systems like this are becoming too complicated to understand all aspects about how they work. You would have thought overriding the TOGA switches should have sounded an immediate alarm though. It seems perfectly reasonable that a very dangerous command or input would be inhibited by the automatic systems, but it seems completely crazy that any command or input would ever be inhibited silently. I'm actually flabbergasted that this is not a fundamental rule of aircraft control systems design at Boeing.
- tass 4y agoAnother poster pointed out that the alarm might also fail to sound, or maybe the switch didn’t actually work, so you need to check it anyway. On Boeing planes, many automations such as TOGO typically have a corresponding physical counterpart - in this case, autothrottle slides the throttle controls upwards to indicate that the engines were asked to go to full power. There’s also the engine instruments which show the engines reaction to the requested throttle level. It’s not hard to know that it didn’t set the throttle to full power, but the pilots didn’t check or didn’t know to check. An alarm would help, but would also be another fallible automation to be relied upon.
- throwawaylinux 4y agoI appreciate there would be indicators to confirm most things, and procedure probably says those checks should be done and alarms should not be relied upon. I still think it's crazy that there aren't alarms for any situation where the airplane disregards an input like this.
- sokoloff 4y agoFolks interested in this topic might also enjoy the AA Training Video “Children of the Magenta Line”: https://youtu.be/5ESJH1NLMLs https://youtu.be/5ESJH1NLMLs Covers advice on when to go from maximum automation towards more manual control of the aircraft when the cockpit gets busy.
- bell-cot 4y agoReaction: The problem is neither automation per se, nor heavy reliance upon automation. The problem is 666-layers-of-intricately-interlinked-shit automation, which no mere human pilot has any chance in hell of understanding the behavior of, in real time, when it is really critical that he do so. Idea: Add a few "Robocide" buttons to the cockpit. If pressed, they deliver a figurative bullet to the brains of the autopilot, dropping the plane into a far simpler "full manual flying mode". Pilots regularly train in doing that, and flying the plane when suddenly dropped to manual.
- dmitrygr 4y ago> Pilots regularly train in doing that, and flying the plane when suddenly dropped to manual. I wish... In fact this is how AF447 crashed... a plane under direct control of pilots... Read: https://www.vanityfair.com/news/business/2014/10/air-france-flight-447-crash https://www.vanityfair.com/news/business/2014/10/air-france-...
- yair99dd 4y agoThank you for the article, an amazing read ill fwd to many
- bell-cot 4y agoAdd a rule - pilots (or cockpit crews) who can't cope with manually flying the airplane don't get to fly. (From Vanity Fair, it sounds like the Air France pilot's union is both extremely powerful, and profoundly hostile to the idea that pilots must actually be competent. Also like opaque layers of both poorly-understood automation and infernally-clever instruments repeatedly got in the way of the pilots doing plausible things to "manually" recover.)
- KMag 4y agoThough, AF447 isn't a simple case of the plane being under direct control. When the two pilots give conflicting inputs, Airbus silently averages out the two inputs. The pilots were confused as to why their inputs weren't seeming to work, then the pilot who was trying to do the wrong thing (pull up) figured out why his inputs weren't having any effect and hit the button to silently ignore the input from the pilot who was trying to do the right thing (nose down). As I remember, it was the more experienced pilot (captain) who was correctly trying to get the nose down, but it seems he never figured out why the plane wasn't responding to his inputs. Also, when they went very far into stall, the stall warning disabled under the assumption it was a sensor problem rather than the airplane actually getting into that bad of a stall. So, paradoxically, starting to improve the situation caused the stall warning to come on, while doing the wrong thing caused the warning to go away. Both pilots were trying to debug with information intentionally being hidden from them. (Honestly, the stick should vibrate or something if your inputs are being ignored or counter-commanded by the other pilot. Better yet, mechanically link the controls as in Boeing planes... it's not great that the stronger pilot wins, but at least both know what's going on.) Granted, there was a lot of pilot error in AF447, but there were multiple user interface issues that greatly contributed to the problem. Edit: Also, as I remember, the start of the incident was that the pitot tubes iced up and the autopilot disengaged itself because it had no idea what to do. It's hard to point to a case where the automation has explicitly given up as a case where we should rely on more automation. Clearly a world where the automation was better would have been better, but just letting the autopilot do its thing wasn't an option. The autopilot disengaged itself.
- wtfmcgrill 4y agoMentour pilot did a detailed video on this accident a couple months ago. He gives the insights on the accident from a pilots perspective. https://youtu.be/Im334Eg3ZLE https://youtu.be/Im334Eg3ZLE
- colechristensen 4y ago>that they don’t fully understand This is it. Humans can understand any system, if you have pilots failing to understand they weren’t trained properly. If you can’t train your pilots properly, use different planes. Nearly every plane crash for decades has been a human failure not a technological one. The way airliners behave is well known and not a mystery at all, if pilots don’t know what to expect its their fault. Designing aircraft for human training needs is indeed important and constantly under review, but I cannot really blame aircraft for the failures of airline training programs.
- pas 4y agoit was basically bad UX :( a nice airframe and a complete shitshow of software and interaction with the machine ensemble
- syastrov 4y agoSimilarly, we have amazingly powerful and reliable computer hardware, but continue to have software that is extremely slow and buggy and hard to use. I theorize that since software is so invisible/intangible, these complexities are not understood by many.
- Mawr 4y ago> But a modern airliner has so many complex automated systems that expecting a pilot to fully understand all of them is unreasonable. Yes... > In both cases, the pilots assumed that the autothrottle would increase thrust during a go-around, but were unaware that they had run up against an edge case where it would not. Also yes, but... > The common denominator between the two crashes was an overreliance on automation. That doesn't follow. The pilots were absolutely right to assume that pressing the go-around button would indeed perform a go-around. The core issue is that the button silently failed. It's like a function that doesn't throw an exception despite failing and instead chugs along in a broken state. It's a simple case of bad UI/UX. If a button that's supposed to do X is unable to do it, the failure should be indicated to the pilots in an obvious way. The recommendations say as much: "The GCAA also issued recommendations intended to make the Boeing 777 a safer aircraft, including that the configuration alarm go off when the throttles are not advanced during a go-around;".
- TheSwordsman 4y agoI think it's a "yes, and..." because it's conceivable that the alert itself could fail to notify pilots for some reason. So the pilots shouldn't rely on the automation to work 100% of the time, nor that the system will alert them to a problem with 100% accuracy. So it's both correct to recommend that the Boeing 777 try to alert the pilots to the misconfiguration, and to reinforce pilot behavior to reduce the risk of being in a position where that misconfiguration happens.
- roxgib 4y agoIt's weird to me that the pilots didn't notice that the engine thrust hadn't increased. I realise they have a lot to monitor and it all happened quite quickly, but it seems like not having your engines at 100% when they're supposed to be would stand out. I'd be interested to see what they would have seen and how obvious it was.
- lazide 4y agoThe engines don’t immediately go to 100% - it takes time for them to spin up, 5+ seconds sometimes. The engines are also very far from the cockpit, and behind them. They hear them, but not well. So it’s not that surprising they didn’t immediately notice the problem. That they did notice, and relatively quickly, is a sign of how well they did. But seconds matter in this scenario, and by then the lag to spin up resulted in no significant change of thrust before impact.
- mjb 4y agoIt's constantly surprising to me how, given how airline safety has improved significantly decade after decade (see https://www.bts.gov/content/us-air-carrier-safety-data https://www.bts.gov/content/us-air-carrier-safety-data, for example), that people still have this "automation is bad" agenda. Every accident is a tragedy, and we should be getting even better, but the narrative that automation (and ironies of automation) are increasing accident rates just seems diametrically wrong. This seems related to the deep belief in risk compensation and risk homeostasis, despite weak evidence for the former and nearly none for the latter. We can be cynical about technology all we want, but to actually improve that needs to be driven by evidence.
- Dalewyn 4y agoAutomation is bad because it can lead to pilot complacency. There are many pilots who wouldn't be able to fly manually, most of whom we only learn about after a mishap due to failed and/or misunderstood automation. Automation is convenient, and in many cases (but not all!) it can making flying safer. But an aircraft with an idiot at the controls is still an unsafe aircraft, regardless how good the automation is.
- jemmyw 4y agoI'm not sure you've made a consistent argument there. Automation is bad but in many cases makes flying safer? MCAS was bad because Boeing and the airlines wanted to retrofit without additional training. It wasn't the automation that was bad per se.
- masklinn 4y ago> MCAS was bad because Boeing and the airlines wanted to retrofit without additional training. And because it worked inconsistently due to terrible implementation.
- Dalewyn 4y agoAutomation can make many flights safer, but this assumes two critical things: 1. The automation is designed and implemented in a reliable, known, safe way. 2. The pilot are competent at flying the aircraft completely manually if necessary. If either of those two assumptions are violated, the flight is not safe. Automation cannot replace incompetent pilots, no matter how good. Pilots can't overcome bad automation if it is not designed and implemented properly (eg: MCAS), no matter how competent.
- deleted 4y ago[deleted]
- shswkna 4y agoThis article comes across as an attempt to frame the narrative in order to exonerate Boeing for what seems like poor quality automation, not at all dissimilar from the MCAS crashes. Other similarities are poor and incomplete documentation and/or withholding of crucial technical information.
- nwallin 4y ago> In the minutes before flight 521 lined up to land, two flights were forced to abort their landings after encountering wind shear; however, they didn’t tell the controller their reasons for doing so, and the controller didn’t ask. The article isn't attempting to make a point here, but for anyone who is unaware, it is this way on purpose. It's not an accident or an oversight or a bad policy. It used to be that pilots had to justify doing a go-around. So in that critical moment, when the pilot is making the go-no-go decision, they're thinking about whether or not they can justify the go-around, not about whether or not the go-around is the right thing to do. There was a tendency for pilots to try to force a landing that just wasn't gonna happen and it resulted in more than a few disasters. So now the policy is to just do the right thing and the pilot doesn't have to explain, not to tower, not to management, not to anyone why they decided to go-around.
- goodcanadian 4y agoWhile you are correct, it is still pretty normal for air traffic control to ask, and it is pretty normal for the pilots to provide a reason whether they are asked or not. Wind shear is definitely information that should be shared for the benefit of other aircraft.
- sokoloff 4y agoI've never been asked the reason for a go-around, but I've definitely offered it if I felt it might affect other aircrews. Interestingly, I have been asked the reason for a rejected takeoff, which I think is ATC policy to ask about and record all RTOs.
- olliej 4y agoIt remains absurd to me that companies making aircraft can include what to me sound like basic safety equipment paid extras. The fact that they had the ability to detect tail- or head-windshear, but charged extra for both is absurd, even if in this case it apparently wouldn’t have changed anything.
- PicassoCTs 4y agoI wish there was a way to emergency lock overhead compartments. In the video several passengers can be seen going for luggage, when it was about life or death..
- pas 4y agothe simple solution would be to tell the passengers that in case of an evacuation there's insurance against material losses, and anyone holding up others will be prosecuted.
- PicassoCTs 4y agoThey are told that, but unawareness of the danger (No mass panic) makes them do the usual debark dance. Just dont offer them access to the luggage.
- pas 4y agono, they are just told to "leave it" at least in the usual safety dance that is always done by the crew has absolutely no mention of sticks and carrots - at least in the EU unless it has changed in the last few months, or if I somehow missed it
- FeepingCreature 4y agoModal interfaces kill again.
- tormeh 4y agoWe must kill Vim.
- cm2187 4y agoI am impressed by the reaction of the passengers. If I was in a plane full of smoke and I saw fire, and the crew wouldn't let me out because they are waiting for an evacuation order from a captain that might be dead, I don't think I would have waited and complied. Smoke kills you way before fire reaches you.
- yks 4y agoNoob question: why did they stow away landing gear so early and did not wait at least until some altitude gain?
- unsafecast 4y agoThe landing gear creates very high air resistances. Certain maneuvers are impossible to do with it down, you can't fly very fast, ridiculously high fuel consumption, et cetera.
- extraduder_ire 4y agoNot sure how much it ads to this write-up, but I remember seeing this incident broken down in a pretty good youtube video by Mentour Pilot. https://www.youtube.com/watch?v=Im334Eg3ZLE https://www.youtube.com/watch?v=Im334Eg3ZLE
- BXLE_1-1-BitIs1 4y agoThe TOGA buttons are usually on the side of the thrust levers. It looks like the reverse levers in the photo are labeled TOGA. The engines were at idle during an extended float; so it could take longer to spool up. The crew knew this, but that's five really important seconds. Definitely should have waited for the kick in the pants before raising the gear. In a heavy aircraft, things happen slower. The airspeed will decay slowly after raising the nose, but you will see some initial climb until gravity has time to assert itself. Increasing headwind will introduce transient increased airspeed masking the decaying energy situation. I wonder why spoilers were not used, but don't know how that fits in with 777 procedures.
- helloworld11 4y agoThis guy's extensive, numerous analyses of plane crashes are downright excellent and deeply detailed like few other texts about the subject that I've ever found online. If he makes mistakes here and there, or attributes blame somewhat incorrectly on occasion, it's a minor issue considering the sheer scope of what he's written. Many of the criticisms in the comments below that I've seen are deeply uncharitable in a way that's so typical of this site, where man y people post dense opinions with all kinds of viewpoints that mis-attribute blame and say mistaken things for all sorts of reasons on numerous subjects. How ridiculously typical of the self-congratulatory audience on HN. Have none of you ever written code that has a flaw or two?