21 ms·
Boeing altered key switches in 737 MAX cockpit limiting ability to shut off MCAS
- JorgeGT 7y agoSo they removed a useful, granular functionality (switching off autopilot trim input while retaining electric-assisted manual trim) and made one switch simply a redundant backup, merging the previously separated functions without clear explanation. A cynical man could be inclined to think that with this change, they casually removed a possibility of pilots/airlines realizing the existence of MCAS: "we have thrown off the right switch to unlink the autopilot from the trim motor and there is still something making automated stabilizer inputs, what could it be?"
- jsjohnst 7y ago> So they removed a useful, granular functionality ... without clear explanation I’m as negative as the next person when it comes to the 737 Max, but this is just a bit to far. Read the rest of the article and you’ll understand a logical reason why this was done and how it was communicated.
- peteradio 7y agoWhat was the reasoning?
- jsjohnst 7y agoThe linked article says why, but to paraphrase, the training has always said to flip both switches. Previously the only time you’d need to flip the switches was due to an electric motor being stuck-on. It’s only with the MAX that there was an actual useful reason to flip one vs the other, but the training was always to flip both and Boeing decided to simplify it. My personal guess is that it was internally totally disparate teams inside Boeing and neither communicated about the change. Is that bad? Certainly. But how often does that happen in big companies? All the time. Sadly it might have had a part in causing fatalities in this case, but doesn’t make it any less common. Either way, it’s not like the change definitively had any bearing. Turning off all electronic control was still the training given to pilots and then reverting to manually turning a wheel that controlled the horizontal stabilizer via cable. It’s not an easy thing to do (turn the wheel manually), especially during a nose dive, but it’s been done before in an emergency and pilots are trained on it. Ultimately, the FAA had a prestigious record due in part to having excellent checks and balances. Whoever decided to move away from DERs to ARs ultimately should get the brunt of this blame, but sadly, that’ll never happen.
- CydeWeys 7y ago> The linked article says why, but to paraphrase, the training has always said to flip both switches. And the root cause reason for that is Boeing was hiding changes in order to make the same type certification. Many decisions were made not in the interest of maximizing safety but in minimizing visible changes. This is one of those. It would have been safer overall to have a new type, and then they could've had a cut-out switch just for the MCAS and trained pilots on how to use it.
- v8engine 7y agoJust to spare others from the search: Before 2004, those Boeing technical employees who worked safety on behalf of the FAA were called “Designated Engineering Representatives,” or DERs. Though paid by Boeing, they were appointed by the FAA and reported directly to their technical counterparts at the FAA. What changed since 2004 is that safety engineers, now called Authorized Representatives, are appointed by and report to Boeing managers. The opaque bureaucratic name for this new structure — Organization Designation Authorization (ODA) — masks the significant change: Instead of having individual Boeing employees authorized as FAA reps, Boeing now has an entire organization within the company so authorized. The individual FAA Authorized Reps — Boeing engineers — report up the chain to their Boeing managers, not the FAA. source: https://www.seattletimes.com/business/boeing-aerospace/engineers-say-boeing-pushed-to-limit-safety-testing-in-race-to-certify-planes-including-737-max/ https://www.seattletimes.com/business/boeing-aerospace/engin... Interesting read.
- jsjohnst 7y agoThanks! Spot on source and agree, it’s a good read.
- SilasX 7y agoSomeone replied but didn't nest it under your comment: https://news.ycombinator.com/item?id=19888261 https://news.ycombinator.com/item?id=19888261
- leetrout 7y agoIt has been fascinating to watch layer after layer peel back around all of this. Bad design decisions forced from chasing the bottom dollar, optional, critical safety features with the warning indicator and now these switches. > Boeing declined to detail the specific functionality of the two switches That's also interesting- I assume they are in CYA mode and wouldn't discuss anything that could be sensitive to the current investigations in any point. I wonder how hard the Seattle Times had to work to get manuals for the MAX. I hope the loss of life is vindicated in the end. I hope there are lessons learned from this but I fear our culture of quarterly profits and lack of real punishment for companies and their directors will result in nothing drastic happening.
- MaxBarraclough 7y agoOne of the lessons learned is that the FAA is no longer a credible institution.
- jeremyjh 7y agoAgreed. Hopefully there is a big shake up and lots of people are sent home without pay. Nothing else could salvage it.
- dd36 7y agoGotta love deregulation. How many institutions are left? Any? CFPB, EPA, HUD, Interior are all run by people that don’t believe in their missions. It’s all about undermining from inside. Who protects the public when private rights of collective action are forbidden by arbitration agreements and the government has been captured by private interests?
- badhombrehn 7y agoThe problem is that only Boeing is left in the US. Lockheed, Douglas, General Dynamics are all out of manufacturing civil airplanes. It's the lack of free market (the decision between Boeing and Airbus is most often a political one), not the lack of regulation that led to this. Boeing is too big to fail now. Regulators are always stupid, but free market is smart.
- salawat 7y ago>A cynical man could be inclined to think that with this change, they casually removed a possibility of pilots/airlines realizing the existence of MCAS: "we have thrown off the right switch to unlink the autopilot from the trim motor and there is still something making automated stabilizer inputs, what could it be?" It's worse actually. They've taken a procedure, and are raising it on a pedestal as if to say "See, this is how a 737 flies! It's all right there! You can stop looking now!" It's as if they don't want anyone to realize that this plane simply does not fly like a 737. Which is what has been getting shouted from the rooftops since the first disaster. What I find wickedly ironic was it was exactly Boeing's insistence to "not inundate" flight pilot's with technical information, and give an accurate accounting of the vast differences in the minute details of the aircraft that created the problem. Someone looked at the checklist item of "both stab trim switches to cutout", didn't look into what that actually meant, and changed the plane to fit the procedure. This change cost extra effort. Instead of using the same switches and wiring they were using before, they actually degraded the granularity of control of the plane to hide the differences. I can't imagine the thought process at the time. Be nice to see what the documentation turns up.
- stingraycharles 7y agoIt's fair to recognize this is not entirely Boeing's fault either: it was the airline industries that don't want to re-train their pilots and wanted this type of plane (American Airlines was specifically keen on getting Boeing to manufacture this plane [1]), so they were kind of told "we want you to make exactly $this, otherwise we switch to Airbus". [1] https://www.vox.com/2019/4/5/18296646/boeing-737-max-mcas-software-update https://www.vox.com/2019/4/5/18296646/boeing-737-max-mcas-so...
- jdsully 7y agoFrom AA’s point of view they were passing on working and proven airplanes from Airbus for something not even on the drawing board yet. For them to agree to this Boeing had to offer them an amazing deal. I don’t blame AA for taking a good offer. They did not force Boeing to do this.
- 7y ago
- exabrial 7y ago> As various warnings went off in the cockpit, they never reached the conclusion to use the runaway stabilizer procedure. We continue to put the microscope on Boeing but this is why there is a person in the cockpit: to oversee the planes automated systems. There were multiple failures, and while we pick apart Boeing, we have not done the same to the pilots.
- eigenvector 7y agoIf a hazard or potential failure mode can be eliminated in the design of a system, it's better to do that than to rely on every single operator to mitigate it individually.
- bumby 7y agoExactly. Engineering the hazard out of the system is a vastly superior way of mitigating it than relying on administrative/operational procedures for mitigation
- AnthonBerg 7y agoThe core of the discussion precisely revolves around Boeing wilfully making it more difficult for the person in the cockpit to oversee the planes' automated systems.
- rootusrootus 7y agoIt would not be invalid to observe that the first Lion Air crew dealt successfully with the malfunction, and ask why the second crew did not. Every aspect of these accidents should be investigated.
- dboreham 7y agoThe first crew had three members.
- zb 7y agoIf I recall correctly, the first crew encountered the issue during cruise, at which point which you have both (a) time, and (b) sufficient altitude that you can afford to push the stick forward in order to reduce the aerodynamic force on the horizontal stabiliser to the point where it will be physically possible to correct the trim without powered assistance. Pilots encountering the issue during takeoff have neither of those things, with results that we've already observed.
- rtempaccount1 7y agoOne thing that appears concerning in the series of articles being written about the 737 Max incidents is that they suggest a culture at Boeing of speed/cost over safety. Now we can assume that the Max won't re-enter service until this issue is completely solved (I'm sure regulators will be extra-scrupulous here and customers will want cast iron re-assurance). However, is anyone going over all the other safety decisions and changes made to other Boeing models over the period of the Max development? It seems unlikely that this was an isolated incident of rushing things for commercial benefit...
- djsumdog 7y agoI'm actually wondering if the 737 MAX will make it back into service. Every single one of these planes will need to be flow back to one of their maintenance centers (like the one south of Seattle) and undergo retrofits. Hopefully it will result in Bowing being required to make this a new plane and a lot of pilot re-certification. If it does return to the market, I wonder if they'll be forced to rebrand it. It's obviously not a 737.
- alexis_fr 7y agoBugs tend to appear in Poisson law of probability: 2 big bugs immediately, 2 others a few weeks later, but that means there are hundreds of them to discover over a period of 5 years (until the D-check). So, in all probability, we don’t even know the list of fatal flaws even yet!
- WrtCdEvrydy 7y agoIt's a 737 with lower engines. The flaw is using software to fix a hardware issue.
- trimbo 7y agoI just wanted to note that Dominic Gates and the Seattle Times have been the gold standard for reporting on the 737 MAX. Really great work by them on reporting a technical subject so non-experts can understand.
- mcsoft 7y agoIt strikes me how a seemingly insignificant decision at the time - to position fuselage slightly higher above the ground - gave Airbus a strategic advantage 50 years later. It forced the competitor into a services of unexplainable and reckless decisions that break basic principles of reliable engineering. Which in turn led to a very visible disastrous business consequences: Boeing had to silently reduce 737 MAX production despite having secured orders for over 5000 planes. A case to be studied in business schools.
- techslave 7y agobasic principles? i think that’s a bit far. the plane isn’t inherently unstable is it? i thought it’s just the black box of MCAS that is the problem and MCAS is there to avoid a lengthy type certification. it’s completely explainable. and it wasn’t seemingly insignificant. it was for fuel efficiency. it was significant and they knew what they were doing.
- mcsoft 7y agoThey had to install MCAS because the plane was unstable in high AoA mode, i.e. during takeoff. And it was unstable because larger CFM engines could not fit under the wing anymore so they had to put it further forward and a bit up. This increased the pitch up moment when a full power is applied. Also, can't really see how the height of the fuselage above the ground affects fuel efficiency.
- hotsauceror 7y ago"Also, can't really see how the height of the fuselage above the ground affects fuel efficiency." It doesn't. They wanted to use a larger, more fuel-efficient engine but the 737's ground clearance couldn't accomodate it. So they moved the engine forward.
- cesarb 7y ago> They had to install MCAS because the plane was unstable in high AoA mode, i.e. during takeoff. Forgive me if I'm missing something, but isn't MCAS disabled while the flaps are extended, which is the case during takeoff and landing?
- djsumdog 7y agoWhat is the endgame now for 737 MAX planes? Every airline using them has had several weeks to totally reorganize to deal with these huge missing parts of their fleets. Boeing keeps talking about fixing them and putting them back in the air, but I dunno. At this point, it should be agreed the fundamental design flaws are serious, and Boeing should have simply made a brand new plane, with much more modern controls and predictable maneuvering. Yes, pilots would have to be certified for these new plans, but that cost pails to the money lost from these grounded fleets. I'd honestly like to see the 737 MAX taken out of service and Boeing simply ending this line. The old planes should be stripped for parts and as much recycled as possible. It's going to cost them a lot of money, and it should. Their mistakes lead to the deaths of two whole planeloads of passengers. Airbus, Bombardier and others will probably make a lot out of this disaster, and that's probably a good thing and will help competition in this small/narrow market.
- cm2187 7y agoA brand new design would probably introduce more serious flaws than an interation of a mature design, so I am not sure it would be safer. In term of public perception, it is a PR disaster but not unprecedented. The DC-10 went through a similar rocky start where passengers just wouldn't want to board it. But after the flaws were fixed it turned out to be quite reliable.
- hirowan 7y ago> A brand new design would probably introduce more serious flaws than an interation of a mature design As evidenced by the ever-crashing 787s and A350s.
- ummonk 7y agoThe 787 was rushed through as well with the same modern Boeing safety culture and we were merely lucky that none of the flaws discovered in the 787 were bad enough to cause fatal accidents before they were rectified. Read up on its battery issues if you want an idea of what could have gone wrong.
- 7y ago
- mannykannot 7y ago'The company said the two switches “were retained for commonality of the crew interface."' ...and without that commonality, the FAA might have required additional training?
- salawat 7y agoGot it in one.
- oldmantaiter 7y agoOff topic, but oh man does that website embody everything that is wrong with display advertising. Left the site open for 6 minutes: 8000 requests, 100MB transferred and a memory footprint of 700MB for that tab and climbing. Absolute garbage.
- chimpburger 7y agoThis article requires more computational resources than a Cray-2 supercomputer (1985). It loads much better in Lynx with a tiny fraction of the resources.
- quickthrower2 7y agoYeah each day I’d feel less and less guilty using adblockers and router rules.
- cletus 7y agoSo a lot is known at this point: 1. The A320neo caught Boeing completely off-guard; 2. The threat of the likes of American Airlines (already a mixed Airbus/Boeing customer and the largest airline in the world) placing a large A320neo order for regional aircraft operations scared the bejesus out of Boeing management; 3. For airlines like Southwest that are pure 737, the prospect of adding a plane that didn't share a common type rating with their existing fleet would complicate their lives and make them vulnerable to a sales pitch from Airbus; and 4. The development cycle for a completely new body was too long for many customers as it would arrive several years after the A320neo. I don't think any of this is in dispute so the constraints for Boeing were to design a more fuel-efficient plane that shared a common type rating with the 737. To get there: - They added newer, more fuel efficient engines. These changed the flight characteristics so they had to be moved; - These engines moving made the plane more vulnerable to a stall situation. To counter this, they added MCAS, which would point the nose down when the AoA sensor told it the nose was too high; - Standard configuration had 2 AoA sensors but MCAS only ever read from one; - There was a safer configuration as an optional extra purchase; - Telling airlines and pilots about this and providing overrides risked the common type rating. I don't believe any of this is in dispute. It is widely believed, but not yet proven, that the primary cause of both fatal crashes was a runaway MCAS that drove the planes into the ground. It's also believed that with proper training a pilot may have been able to counter this (as happened the day before the Lion Air crash with a pilot in the jump seat). Now what I find interesting is the response people have to all this. Some claim this is a fundamental design flaw that puts a shadow over the plane. Others believe it will be corrected and everything will be fine. I'm firmly in the first camp: the plane CLEARLY flies differently to a 737. An automatic system, with no triple redundancy, was required to correct the flight characteristics of the plane. I'm no expert but it seems to me the plane is fundamentally flawed at this point.
- Obi_Juan_Kenobi 7y ago> These engines moving made the plane more vulnerable to a stall situation. I don't think this is correct. A lot is being made about the engine move with regard to aerodynamic behavior, but it doesn't seem at all outside of the bounds of what is considered standard. Pilots report that a similar 'light stick feel' at high AoA is already present in aircraft like the 757. It's not that the aerodynamic change was worse or better, it's simply that it was a change at all. MCAS was there to satisfy the type rating. MCAS is not anti-stall. The crucial mistake appears to be the extreme failure mode of the system; it is permitted to input high stabilizer trim angles without limit. Without any kind of restriction, the failure goes from annoying (pull up on flight stick + fiddle with stab control until you solve it) to the deadly crashes we've seen where pilots are in extremely tricky situations. The fix could be as simple as making MCAS cutout in the case of an AoA disagree (no real hardware changes here) as well as limiting the input of extreme trim angles.
- jordache 7y agofack man, this is some crappy vendor releasing un-documented APIs.. Except this is a machine that is carrying hundreds of lives. Regulations needs to ensure every little detail about these planes are articulated to the end-user.
- breatheoften 7y agoTaking a forward view — I think FAA needs a way to defray the impact of operational costs by incentivizing the uptake of changes that produce safety improvements. The ultimate economic reasoning for the 737 MAX was entirely to reduce operational cost: “give us a plane with better fuel efficiency that we don’t have to pay to train pilots for!”. This is actually a deeply broken incentive that, I think this case shows, should not exist. The plane delivered is objectively far worse than it _could be_ were it not for the need to reduce operational cost. Ultimately — some large portion of that operational cost is induced as a result of the need to comply with safety regulations. That creates downward pressure on unecessary experimentation but also has created this very perverse incentive to design planes that are _deliberately_ not as safe as they could be given application of some approximation to “our best engineering knowledge”. Pretty early on in the design process for the 737 MAX — there shouldve been a meeting with regulators. The regulators should’ve been presented with this option and the design and economic reasoning for it — and they should’ve also been presented with another option wherein there was a relaxing of the need to defuse operational costs. The regulators should be empowered to then say “don’t build this broken plane, build the safer plane and we will introduce tax credits sufficient to help offset the airlines retraining costs.”
- cyberferret 7y agoWait, what? Reading this is turns out that flipping both switches on the centre panel not only deactivates MCAS, but also deactivates the thumb switch on the control wheel that the pilots routinely use to change the elevator trim. So, MCAS trims the plan to a dangerous attitude, then when pilots follow Boeing's SOP to disable, they lose the ability to re-trim the way they have done it a million times before? I can see valuable second being eaten up when pilots flick the thumb switch towards them and sit there wondering why the back pressure isn't easing up. And the report also says that the manual trim adjustment wheel would be harder to move due to the load on the tail when the aircraft is in a nose (death) dive? This is already on top of the fact that the pilots have to hold around 60Kg back pressure on the control wheel while leaning forward to crank the manual trim wheel? Recipe for absolute disaster.
- cjbprime 7y agoYes, exactly. The MAX "stab trim cutout" simply disabled the trim motor altogether. Everything you describe appears to have happened in the Ethiopian flight report. A 737 pilot also tried to run the same flight in simulator and had his arms wrapped around the control column trying to hold level while the copilot failed to move the trim wheel.
- rasz 7y ago>737 pilot also tried to run the same flight in simulator https://news.ycombinator.com/item?id=19627660#19627755 https://news.ycombinator.com/item?id=19627660#19627755 That video was removed :(
- sabujp 7y agoI don't understand why there's not something that just allows the pilots to completely turn off all autopilot systems without also turning off hydraulic supports? It seems like if they turn off autopilot it makes the steering become not powered or something?
- deleted 7y ago[deleted]
- Jabbles 7y agoWhat changes should be made at the FAA (or international equivalents)? Clearly they weren't able to do their job of ensuring the aircraft were safe. They must share some of the responsibility, but how could we improve them?