18 ms·
How the Boeing 737 Max disaster looks to a software Developer
- crimsonalucard 8y agoThe irony of software is that it is the only discipline in engineering where results can be proven with logic. Yet we still do testing on it as if it was a blackbox.
- jsiepkes 8y agoWell we can prove some things behave under certain conditions in a certain way. We can't prove the absence of bugs.
- crimsonalucard 8y agoNo, we can literally prove it correct based off of a specification. https://en.wikipedia.org/wiki/Correctness_(computer_science) https://en.wikipedia.org/wiki/Correctness_(computer_science) In science nothing can be proven but in the world of logics and math, things can be proven. Bugs can arise where programs intersect in the real world.
- softwaredoug 8y agoYeah and then the specification at that level of specificity IS the code. How do you prove the specification is bug free?
- crimsonalucard 8y agoDefine what is a bug in a spec? I defined the program to do one thing and one thing only. What does does it mean when I have a bug in my definition? There's a definition for the definition? Makes no sense.
- aassddffasdf 8y agoIt means that the one thing you defined it to do was not the right thing.
- throwaway2048 8y agoYou can argue that no code ever has bugs by that logic, after all you defined the program to do one thing, and it did it, it was merely human expectation that was in error.
- crimsonalucard 8y agoThat's what I'm arguing. Code can be proven correct against a formal spec. However, because we don't do proofs in software, there are bugs.
- throwaway2048 8y agoCode is itself a specification.
- crimsonalucard 8y agoIt can be, but generally it is not.
- tluyben2 8y agoIf the spec is complex you are probably proving the bugs in the spec are indeed correctly implemented... Edit: that said, spending the formal spec time will probably reduce the number of bugs far below than what we find normal now. But money...
- AnimalMuppet 8y ago> Bugs can arise where programs intersect in the real world. Well, the context is flight control software for airplanes. If that code is bug-free but doesn't intersect the real world, that's rather useless. And if it intersects the real world but therefore is not bug-free, that's not a great argument for proofs of formal correctness. But all of that is kind of beside the point. The MCAS specification was the wrong thing. Proving the implementation correct is useless. (You asked what is a bug in a spec? MCAS shows you the answer. Only taking input from one sensor is the wrong thing. Repeatedly applying nose-down is the wrong thing. It seems like I'm missing one or two more.)
- crimsonalucard 8y agoYou can prove the code correct against the spec. However you can't prove that the spec will do what you expect it to do because the spec is where the intersection with the real world occurs. Therefore, you must test the spec. Everything that falls under the spec can be formally verified to 100% correctness.
- mehrdadn 8y agoIsn't digital logic testing a thing for hardware?
- Gibbon1 8y agoComplex hardware often contains bugs many of which map to software bugs. Hardware though tends to be highly constrained in how it's operated. Man: Doc it hurts when I bend my knee this way. Doc: Well don't do that. EE: The hardware state machine runs into the weeds when given these inputs. Tech Rep: Yeah don't do that.
- bollu 8y agoI'm not sure you understand how difficult it is to prove software correct. I've written a decent amount of Coq code. It's quite bonkers how much of proof one needs to write to get anything done. For reference, the certified compiler CompCert's code base is something like 10% code and 90% proofs.
- crimsonalucard 8y agoNo, I get it, but still... it's ironic. Software for a plane, though, I would want 1% code 99% proofs.
- scrape_it 8y agoWell, to support the top parent's comment, here's what John Carmack had to say about SAAB's fighter jet: > The fly-by-wire flight software for the Saab Gripen (a lightweight fighter) went a step further. It disallowed both subroutine calls and backward branches, except for the one at the bottom of the main loop. Control flow went forward only. Sometimes one piece of code had to leave a note for a later piece telling it what to do, but this worked out well for testing: all data was allocated statically, and monitoring those variables gave a clear picture of most everything the software was doing. The software did only the bare essentials, and of course, they were serious about thorough ground testing. > No bug has ever been found in the “released for flight” versions of that code. so not quite that tests are useless because blackbox, but more that yes, it is possible to write software that facilitates showing a code will do what it says it will do. In another note, I do feel sorry for the engineers but at the same time Airbus also has similar MCAS system, but the only thing that saves them is an extra Angle-of-Attack sensor whereas I believe the boeing relied on just one or two. Airbus had 3 AOA sensors and if 2 agreed, that was the data fed into the MCAS. It still boggles my mind that they would place a bigger engine when the original plane was not built for it at all, this was 100% profit orientated move to prevent Airbus from taking Boeing's majority marketshare, it's highly ironic that the opposite results. Even more infuriating is that Boeing passed it off as the exact same plane, not much extra training is necessary, along with FAA giving their blessing, now caught up in the mess. Canada for instance is now looking to the EU for an independent auditing, as any credibility FAA has built up over the past years has taken a significant hit.
- alacombe 8y agoProving something is correct is only part of the problem. A random bit flip (no matter amount of error correction and physical hardening) and your proven-correct software goes amok. And this is just one of the many issue you can encounter dealing with the physical world.
- crimsonalucard 8y agoYeah, you're right. Can't completely get rid of tests but the irony still exists.
- samatman 8y agoIronically, the solution to this in spacecraft, where radiation is a given, is 2 of 3 with polling. The system which should have been using for attitude detection in the 737 MAX, and wasn't.
- throwaway2048 8y agoDosen't eliminate a single point of failure, after all a bitflip in the voting mechanism can also result in error.
- WalterBright 8y agoProving that software matches the spec is one thing, proving that the spec is free of bugs is quite another.
- bdamm 8y agoIndeed. It is normal that implementation will find a bug in the spec. Anecdotally, have never found a flawless RFC. Every real-world implementation of an RFC I've ever done has found some discrepancy when it came to compatibility testing.
- crimsonalucard 8y agoYes. But for critical software systems, what excuse does one have not to prove that the software does not match the spec. There is no excuse not to test the spec, but what excuse is there not to prove the software?
- teddyh 8y agoI refer you to Joel Spolsky about formal proofs of programs: […] So in the first day of that class, Dr. Zuck filled up two entire whiteboards and quite a lot of the wall next to the whiteboards proving that if you have a light switch, and the light was off, and you flip the switch, the light will then be on. The proof was insanely complicated, and very error-prone. It was harder to prove that the proof was correct than to convince yourself of the fact that switching a light switch turns on the light. Indeed the multiple whiteboards of proof included many skipped steps, skipped because they were too tedious to go into formally. Many steps were reached using the long-cherished method of Proof by Induction, others by Proof by Reductio ad Absurdum, and still others using Proof by Graduate Student. For our homework, we had to prove the converse: if the light was off, and it’s on now, prove that you flipped it. I tried, I really did. I spent hours in the library trying. After a couple of hours I found a mistake in Dr. Zuck’s original proof which I was trying to emulate. Probably I copied it down wrong, but it made me realize something: if it takes three hours of filling up blackboards to prove something trivial, allowing hundreds of opportunities for mistakes to slip in, this mechanism would never be able to prove things that are interesting. https://www.joelonsoftware.com/2005/01/02/advice-for-computer-science-college-students/ https://www.joelonsoftware.com/2005/01/02/advice-for-compute...
- nextos 8y agoI have a MSc in formal methods, and this is really misleading. Surely you cannot make some proofs easily. However, how did Airbus verify that some errors simply do not exist in their fly-by-wire software (which is ~100 KLOC implemented in a subset of C)? (I'm sure Boeing also employs these techniques internally). Using abstract interpretation. There are tons of formal methods, ranging from type systems to formal proofs. Lots of compromises can be made to make them practical and useful for a particular domain. Look into [1,2] for some quick introductory examples. Going straight into formal proofs is in general a really bad idea. I have worked on railway control systems, with really nasty potential race conditions and managed to prove the absence of large classes of errors. Then derived implementations formally. It's really not that hard. There's even a subfield of CS looking into verifying formal properties of biological systems [3], which are really complex. [1] http://adam.chlipala.net/frap/ http://adam.chlipala.net/frap/ [2] http://www.concrete-semantics.org/ http://www.concrete-semantics.org/ [3] http://lucacardelli.name/Papers/Abstract%20Machines%20of%20Systems%20Biology%20%28Draft%29.pdf http://lucacardelli.name/Papers/Abstract%20Machines%20of%20S...
- threatofrain 8y agoMathematicians often don't rely on the exact correctness of a proof's description, instead relying on a rough sketch of strategy.
- wpietri 8y agoWhat a stellar example of an article on a complex topic written to be clear enough for the audience to understand. I especially like the way he brought it back repeatedly to hands out the window and bitey dogs.
- wpietri 8y agoI also heartily agree with him that software's general laxity with regards to reliability is contagious. I've come think that calling it all "software" is dangerous, like thinking of all things made with atoms as the same. I think we as an industry should get together, divide the work into various domains, and establish professional and ethical standards for the domains that matter. Standards with teeth, such that developers who want to do the right thing in the face of bosses insisting otherwise have the backing of their peers. And also such that developers who don't care about the right thing fear the professional consequences. I honestly think this is something we should do regardless. But as a practical matter, if we don't governments will do it for us soon enough. Software keeps getting more important, as do its failures.
- ubertakter 8y agoSomething like (or exactly) software systems safety methods should be applied to critical systems (such as aircraft systems). DoD does this for all of their critical software. And I say "something like" only to indicate there may be something particular about software in aircraft. I doubt it though. And as the article indirectly points out, analysis was severely lacking. DoD Software System Safety handbook https://www.acq.osd.mil/se/docs/Joint-SW-Systems-Safety-Engineering-Handbook.pdf https://www.acq.osd.mil/se/docs/Joint-SW-Systems-Safety-Engi... Full disclosure: The company I work at does this type of work. I don't work in that group.
- xiphias2 8y agoWe already have standards, one of which is clear technical documentation of the system, which the users were not getting. Also we have standards for writing fault tolerant systems (I would say even 2 measure devices are not enough, you need at least 3 to be able to decide). In Boeing it's clear that software best practices and standards were not followed at all. As an example here's the description of the Google Test Certificstion process: https://mike-bland.com/2011/10/18/test-certified.html https://mike-bland.com/2011/10/18/test-certified.html
- HankB99 8y agoI'm unclear on how moving the engine up causes application of power to cause the attitude to tend to go nose up. I would expect the opposite to happen. I think something else must have changed such as the center line of the engine relative to the center line of the plane. Or did moving the engine forward at the same time cause this effect?
- fwip 8y agoThey moved it forward so they could move it up, but the centerline of the engine was still under the original engine's centerline.
- leoedin 8y agoI don't think it does. Other sources I've read suggest the larger nacelle introduces a pitch up during certain flap configurations and at a high angle of attack. It's not the engine thrust that's the problem, but the drag due to the large and further forward nacelle. That's the justification for the MCAS software. https://theaircurrent.com/aviation-safety/what-is-the-boeing-737-max-maneuvering-characteristics-augmentation-system-mcas-jt610/ https://theaircurrent.com/aviation-safety/what-is-the-boeing...
- ncmncm 8y agoIt's both. The engine thrust center is also farther below the center of pressure of the airframe. And it can apply a lot more thrust than the original engines. The point about lift from the engine nacelles was that it acts as positive feedback: if you are already pitched too high, it acts to worsen the problem. And, because it is farther forward, it has a greater lever arm to act.
- HankB99 8y agoThanks, Those factors make sense.
- matt4077 8y agoForward, where the wing is higher off the ground. The rest is like a skateboard suddenly accelerating, which will tend to pitch you nose up (and on the ground).
- murkle 8y ago> In the 737 Max, only one of the flight management computers is active at a time—either the pilot’s computer or the copilot’s computer. And the active computer takes inputs only from the sensors on its own side of the aircraft.
- paulopontesm 8y agoAlso wondered what was the logic behind this decision...
- nutjob2 8y agoLike everything else, they were maintaining certification. The original avionics worked that way. The same reason that the LCD screens display simulations of the original avionics hardware such as the artificial horizon.
- digikata 8y agoI suspect combined reliability of a simple switch and two independent systems is likely higher than one composite system and pilots or software trying to estimate and select which combinations of computers & sensors are "good" in the middle of an emergency.
- airbreather 8y agoAnd on the surface this looks like a reasonable comment, but it is exactly why there is a whole branch of engineering dedicated to understanding how to build safer systems. Counter-intuitive results abound. So many issues - a simple switch usually has poor diagnostics, at least in one mode of failure, so you dont know it has failed until it is too late. A continuous measurement device connected to a computer/s will have a vast array of available diagnostics, 'most probably leading to less "dangerous undetected failures" than a simple switch, or combination of. And "independent systems", sounds easy, but in practice full independence is almost impossible to achieve, and messy unpredictable humans dominate the common cause failures that overlap these systems. There is more, much more, but this is why it is hard to right readable articles about these things, so much devil is in the detail that is hard to explain in bite sized portions.
- laydn 8y agoThe article states: "When MCAS senses that the angle of attack is too high, it commands the aircraft’s trim system to lower the nose. It also does something else: It pushes the pilot’s control columns downward" No, it does not push the pilot's control columns downward.
- Declanomous 8y agoI think that the author is mistaking the fact that trimming the plane forward means that more force is required to pull back on the yoke for the plane actively pushing the yoke forward.
- cjbprime 8y agoYeah, maybe also some Boeing/Airbus confusion. The author doesn't seem to understand the extent to which the 737 cockpit is mechanical, with cables running to control surfaces. It's an important point because it likely explains why the Ethiopian pilots were unable to regain control after disabling electric trim, becoming overpowered by aerodynamic load on the stabilizer.
- mfer 8y agoIn newer 737 versions it's no longer cables... http://www.boeing.com/commercial/aeromagazine/aero_02/textonly/sy01txt.html http://www.boeing.com/commercial/aeromagazine/aero_02/texton...
- ReGenGen 8y agoFYI That doc is about "Propulsion Control Systems" not elevator trim or control surfaces.
- _bxg1 8y ago"Various hacks (as we would call them in the software industry) were developed." Dear God. That's a sentence I never, ever wanted to hear about an aircraft.
- nutjob2 8y agoThe modern 737 is a collection of hardware and software hacks, flying in close formation.
- cmauniada 8y agocloud formation* Sorry, I couldn't resist.
- penagwin 8y ago> flying in close formation "conveniently in close proximity most of the time, usually". I wonder what the equivalent of "Developing in Production" looks like for planes, or if this is it?
- WalterBright 8y agoSo is any airplane that was ever built that is capable of flying.
- WalterBright 8y agoAll engineering designs are full of compromises (the article uses "hacks" meaning compromises), as there are a large number of competing issues at work. Pretty much none of those issues are ever aligned along the same axis. For just a taste of this, an airliner flies at high altitude, and at low altitude. It flies at low speeds, and high speeds. It flies heavily loaded and empty. Optimizing for any one of these regimes means unacceptable behavior on the others. So compromises are necessary. Ever notice the flaps on the wings? They're a compromise (a "hack" if you will) to change the shape of the wing to make it work better across different flight regimes. Because metal isn't very flexible, and a long list of other issues with the machinery that operates the flaps, the shape of them is hardly anything but compromises.
- rkagerer 8y agoI believe the relative ease — not to mention the lack of tangible cost — of software updates has created a cultural laziness within the software engineering community. -- This --^ As someone who carefully crafts their code to strive for perfection, seeing sloppy work out there in the wild drives me nuts. I know folks here will deride me for being "inefficient", but in the long term I still maintain from my experience it's less efficient to push out buggy software and try to fix it retrospectively.
- rubicon33 8y agoIf only managers and CEOs saw it the same way.
- winrid 8y agoAt my current company all leadership is in agreement that it is more expensive to deal with bugs in production.
- Frost1x 8y agoBut CD...agile... It really depends on the scenario. If you can bill those fixes to the client or if you have to eat the cost to make the fixes. Most modern development processes seem to push this model simply because they can essentially pass the costs to the client and charge other or future clients for the improvements.
- winrid 8y agoWe charge upfront for someone to use the product and negotiate any feature requests.
- WalterBright 8y agoYou'll see it in any piece of software written by any developer who is self-managed. The notion that software quality would greatly improve without managers is a myth.
- ggm 8y agoDNS: changing the engines mid-flight. we've used this analogy.
- salawat 8y agoCouple pedantic quibbles, forgive me: >I will leave a discussion of the corporatization of the aviation lexicon for another article, but let’s just say another term might be the “Cheap way to prevent a stall when the pilots punch it,” or CWTPASWTPPI, system. Hmm. Perhaps MCAS is better, after all. It's not actually just intended for when pilot's pour on the thrust, it's for any time they pour on the AoA. There need not be any throttle change involved to bring this about. The most readily imagined example, however, is low speed, large throttle increase like you'd have if you were aborting a landing attempt to go around. >When MCAS senses that the angle of attack is too high, it commands the aircraft’s trim system (the system that makes the plane go up or down) to lower the nose. It also does something else: It pushes the pilot’s control columns (the things the pilots pull or push on to raise or lower the aircraft’s nose) downward. Nothing I've read characterizes MCAS as having an inbuilt-stick pusher. I've mentioned before that MCAS shares a spot with stick-pushers in terms of what they are trying to do, and problem they are employed to solve but MCAS does not actively put force on the stick through an extra mechanism with the intent to actuate a control surface deflection, and alerting the pilot through the haptic response, which is the defining characteristic of that type of system as I understand it. All MCAS does is modify the trim, which has the effect of passively modifying the flight characteristics of the aircraft. >In the 737 Max, like most modern airliners and most modern cars, everything is monitored by computer, if not directly controlled by computer. In many cases, there are no actual mechanical connections (cables, push tubes, hydraulic lines) between the pilot’s controls and the things on the wings, rudder, and so forth that actually make the plane move. And, even where there are mechanical connections, it’s up to the computer to determine if the pilots are engaged in good decision making (that’s the bitey dog again). As far as I am aware, Boeing has maintained manual reversion with regards to the trim system and yoke in the 737 MAX 8. I.e. there are direct connections to the control surfaces from the pilot's hand actuated controls. Boeing's concession to FBW is by implementing parallel automation managed control circuits that allow for electrically driven manipulation of control surfaces fed back to the pilot through the mechanical linkage. The envelope-protection -> bitey-dog analogy in this case, is still an accurate characterization. >But it’s also important that the pilots get physical feedback about what is going on. In the old days, when cables connected the pilot’s controls to the flying surfaces, you had to pull up, hard, if the airplane was trimmed to descend. You had to push, hard, if the airplane was trimmed to ascend. With computer oversight there is a loss of natural sense in the controls. In the 737 Max, there is no real “natural feel.” As far as I can ascertain, there still is "natural feel" in regards to pitch control. There are no hydraulic boosters in place. The problem though, is that MCAS actually hides the aberrant behavior at high AoA from the pilot by intentionally down-trimming the plane. This gets the job done (by some definitions), but does it instead through the trim system (which operates automatically in other circumstances enough where the pilot may mistake the behavior for some other system) and without doing anything to the stick (the primary instinctual control for the plane). Other than those quibbles, this is a beautiful write-up that is a treat to read. The Supplemental Type Certification section was also informative for me, as it illustrates unambiguously that there was a process to handle this exact type of augmentation which was sidestepped by Boeing for whatever reason. Bravo!
- ReGenGen 8y agoThe 737 Max keeps getting viewed as an "Engineering Failure" but we should consider if this really was a "Management" failure. The unstated goal w/MCAS was to avoid additional pilot type certification. (If you tell pilots about MCAS, give them an off switch, or if the system switches off... then pilots need to be trained specifically on the MAX.) If management gave the MCAS project to Senior Boeing Engineers they would likely push back jeopardizing unstated goals. Executives likely steered this project to junior engineering or yes-men... who delivered a solution which avoided pilot training.
- theclaw 8y agoThe article states that MCAS was implemented "on the hush-hush," which makes me wonder if it could even be subject to the same level of quality control as other features of the software. It might have had to bypass some of the more stringent parts of Boeing's development process to avoid appearing in documentation that the customer or the FAA might see.
- ncmncm 8y agoThis appears to be what happpened. If so, it amounts to criminal negligence. People should go to jail, but if anybody does, it will certainly not be the ones ultimately responsible. Most likely Boeing will pay fines and court judgments, something probably already factored into their stock price, impacting people holding the stock this year, not those who might have demanded better management five years ago. Certainly the whole top tier of management should be fired, but that won't happen either.
- jeremyjh 8y agoHow about the fact that one plane full of people going down was not enough to wake everyone up. No, we need two planes to go down, and still the FAA was telling us we have nothing to be concerned about. The biggest problem here is not the engineering or even the management. Its regulatory capture.
- _bxg1 8y ago
- _bxg1 8y ago"Neither such coders nor their managers are as in touch with the particular culture and mores of the aviation world as much as the people who are down on the factory floor, riveting wings on, designing control yokes, and fitting landing gears. Those people have decades of institutional memory about what has worked in the past and what has not worked. Software people do not." I wonder if, once basic programming becomes a part of standard education, these kinds of problems will be mitigated. Right now if you're a developer, you're not anything else. So developers come in as outsiders to any given industry and have to learn about it. But software is applicable to nearly every industry, so we end up with lots of domain-specific code written by people who don't know those domains very well. What if the domain experts could not only speak the language of code, but could do some of the coding themselves?
- Sharlin 8y ago> What if the domain experts could not only speak the language of code, but could do some of the coding themselves? Some can, especially in the academia. It… sort of works but is certainly not optimal. The truth is, it is not enough to be a coder who knows something about the domain, and neither is it enough to be a domain expert who can code. You have to be an expert at both, like the author of TFA. Or at the very least, your team has to be.
- _bxg1 8y agoCould there not be domain experts who know how to express their general ideas in terms of code (pseudocode, even), paired with coding experts who can architect the overall software system? You'd still need software experts, but you wouldn't have just software experts.
- Sharlin 8y agoYes, as I said: teams are a superorganism, a force multiplier. But a team of coders isn't enough, you need a close-knit team of people with diverse areas of expertise, and those people need to be great at communicating.
- 8y ago
- NoNameHaveI 8y agoSo, the linked article is an updated version of an earlier article in the EE times. In the comments section, a reader (ie) Frankly, I am astonished that a single point of failure (AOA) could make it through a FMEA (Failure Mode Effect Analysis). For those who are unfamiliar, a FMEA is (basically) looking at each part of a system and saying "What happens if it breaks?". Having worked in commercial vehicle software development, I am astonished as well.
- theclaw 8y agoI don't work in this field. Do you have to publish the findings from this analysis? Is it possible that no such analysis was done because the MCAS system had to be kept quiet?
- cjbprime 8y agoYes, you do, as part of certification. Boeing misrepresented the degree of control MCAS has and was able to avoid making it a redundant system as a result. Layperson's description here: https://www.seattletimes.com/business/boeing-aerospace/failed-certification-faa-missed-safety-issues-in-the-737-max-system-implicated-in-the-lion-air-crash/ https://www.seattletimes.com/business/boeing-aerospace/faile...
- cjbprime 8y agoIt appears it made it through because Boeing didn't reveal that MCAS can incrementally mistrim until it has overpowered the pilots. They may even have added that capability after going through the initial certification.
- wnevets 8y ago>Unfortunately, the current implementation of MCAS denies that sovereignty. It denies the pilots the ability to respond to what’s before their own eyes. >In the MCAS system, the flight management computer is blind to any other evidence that it is wrong, including what the pilot sees with his own eyes and what he does when he desperately tries to pull back on the robotic control columns that are biting him, and his passengers, to death. Wow, I had no idea the issue with the 737 Max was so nuts. I can't imagine what the pilots were going through when this was happening.
- ncmncm 8y agoWell, no robotic control columns, but the effect of silently adding uncommanded trim is the same.
- ncmncm 8y agoThis is a very good analysis, but fatally incomplete. One really essential reason those planes crashed was that each time the MCAS triggered, it acted like it was the first time. If it added 1 degree of trim last time, it adds a second this time, a third next time, up to the five degrees that runs the trim all the way to the stops. A second reason is that, under the design still on file at the FAA, it could only add a maximum of 0.8 degrees (each time). This was raised to 2.4 degrees after testing, so only two hits could, in principle, put you almost to the stops. A third was that the only way to override the MCAS was to turn off power to the motor that worked the trim. But above 400 knots, the strength needed to dial back the trim with the hand crank was more than actual live pilots have, especially if it is taking all their strength to pull back on the yoke. A fourth was that, with two flight control computers, the pilot could (partly) turn off a misbehaving one, but there is no way to turn on the other one. You have to land first, to switch over, even though the other is doing all the work to be ready to fly the plane. A fifth was that it ignored that pilots were desperately pulling back on the yoke, which could have been a clue that it was doing the wrong thing. A sixth was that, besides comparing redundant sensors, it could have compared what the other flight computer thought it should be doing.
- AWildC182 8y agotl;dr the first thing you learn doing control systems is NEVER use += 1; when it relates to an electro-mechanical device. I learned this a long time ago by putting a robot through a door-frame at max speed. Edit: To add to this in a way that might actually make it useful to someone, motor outputs should almost always be a continuous function of something rather than their own internal state. MCAS should have been coded as something like (vastly oversimplified) if(AoA >= MAX_AOA) trimPosition = g(f(AoA, IAS), trimPosition, dt);//AoA/IAS dependent ramp function else trimPosition = g(trimInput, trimPosition, dt);//ramp function not if(AoA >= MAX_AOA) trimPosition += 1; Because the latter is very likely to result in a runaway even if you have bounds checking somewhere else. Worst case you don't have bounds checking and the position value/register value loops over and the tail just starts spazzing out, flapping up and down as fast as the motor will drive it.
- 8y ago
- rdiddly 8y agoDespite the possible negative consequences for our salaries, we need to work to remove the divide between the software people and the subject-matter experts. Those who use, but don't necessarily build, software, seem to place undue trust in "the computer" to be always right, whereas we all know the computer is just being instructed by ordinary human schmoes like us. And we (the schmoes) are reliant on what to me seems like a tiny bottleneck of 2-way communication with the SMEs. It doesn't need to be binary technical/non-technical; everybody is, to some degree, technical. If you can use a knife, you're technical. And it doesn't need to be this rigid specialization, I do computers / I do aviation. If we removed that divide we might start to widen and deconstrict that bottleneck, and we might even start to find more people like this author who know both, and who don't need to have an aviation SME tell them to check more than one sensor for example, because when they're writing it, and coding the part where you set or decide the value for AOA, they automatically say to themselves "hey let's add a cross-check here like I always do when I'm flying." Specialization seems like a magic bullet but it also creates a big burden of oversight & communication, I guess is my point. It's like splitting the monolith of the human world into microservices, and with the same resulting problems.
- amluto 8y agoOne thing I really don’t get about MCAS: even disregarding all its bugs, it seems like a really awful style of envelope protection. I’m neither a pilot not an expert, but I can imagine several more reasonable strategies when the pilot pulls up too hard: ignore the problematic yoke input, offset the elevators a bit, or apply more force to the yoke. Moving the stabilizer out of trim seems totally wrong. As an analogy: almost every car has envelope protection. But no car designer in their right mind would build an ABS system that responded to wheel lock by moving the brake pedal or loosening the master cylinder (and not putting it back!). Similarly, it would be crazy for a stability control system to counter oversteer by offsetting the steering wheel (and leaving it offset).
- cjbprime 8y agoElevator probably isn't powerful enough to avoid the impending stall once you get there.
- cjbprime 8y ago... and the 737 MAX control surfaces are mechanical. I don't think there is any ability to ignore yoke input! It's connected to the elevator with a cable. This isn't an Airbus-style fly by wire aircraft where input signals are being interpreted as suggestions for actions for a computer to take.
- adreamingsoul 8y agoSomething that stood out to me in this article was the author's opinion of how software developers are removed from the subject matter. From my experience I tend to agree but I hope we can change this. For context, I'm a software engineer and designer with a UX focus. As part of my process to design a system, interface, feature, or fix I first immerse myself into the world of the person who is using the software. What's the environment that these people are working in? What are their motivations and frustrations? What allows them to be successful in their job? Over the last couple of years I've had less and less time to answer those questions. Either because of a high workload or because management didn't see the value in spending time to research the people using our software. That trend is alarming and concerning to me. If we as software engineers are not aware of the people who are using our software, how are we solving the right problems?
- linuxftw 8y agoWe're agile now. We don't have a solid view of the problems we need to tackle, we're just given a little user story that says "Plane noses down with these inputs" and we code it up, send it over the wall. The larger the org, the less likely the actual engineers that will design the system are invited to the planning and arch meetings. So when stupid ideas come up in those meetings, there's no one to say "no, that's dumb" because management only knows how to say yes to their boss.
- mch82 8y agoThe Agile Manifesto calls for frequent conversations and demos with customers so developers understand customer needs and acceptance criteria. If you’re given a story card & there is no customer interaction, then there’s a chance the team is calling itself Agile without actually using Agile.
- mch82 8y agoYou’re asking great questions. Keep fighting for the time you need to answer them. I’m 13 years out of college & my perception is companies care more about stakeholder research than they did when I started working, so I think the overall trend is better.
- sisu2019 8y agoSorry, I don't believe that our perspective on this is especially valuable or insightful. Everyone with an average IQ can explain in 5 sentences what went wrong with this plane. Yes new engines didn't fit the plane, fixed with software, well done. But why? It's not profits. Boeing has been for profit from the start. It's something more. We are getting bad at doing hard things. Here in Germany we can't seem to finish a new airport (BER). Or a new train station (Stuttgart 21). Or the new ICE trains. Or keep the autobahn bridges maintained. And software? There is an article about problems with Win 10 updates ever day now it seems. And what about all these new SPA type websites that pull down 30 megs of javascript in order to keep failing at basic tasks in new and exciting ways? On latest android there is a packet fragmentation bug that has gone unfixed for 6 month and counting now. Oh it only prevents people from using amazon and netflix so no big deal. I grew in a time where we expected stuff to get better and better but lately it seems we'd do well to just hang on to what we have.
- theclaw 8y agoIt was profits. Airbus did effectively same thing with the A320 Neo in 2010, and it became the fastest selling commercial aircraft in history [0]. Boeing clearly was unhappy with that and needed to do something to compete. [0] https://en.wikipedia.org/wiki/Airbus_A320neo_family#Orders_and_deliveries https://en.wikipedia.org/wiki/Airbus_A320neo_family#Orders_a...
- sisu2019 8y agookay let's back up and remember that you can't compete with a plane that crashes all the time. So the reason is clearly not profit, in fact they will lose a big chunk of money from this and that wasn't hard to predict at all.
- bumby 8y agoI think it's a mistake to imply the managers making this decision had our hindsight knowledge. Of course if they knew this would happen they would've taken a different course, for profits and other reasons. To your larger point, I think as systems get more complex it's much more difficult for management to make accurate risk/benefit decisions. Take the Shuttle Challenger disaster. In Feynman's report, the management estimated something along the lines of a 1-in-100,000 chance of catastrophic failure. I think the actual number was ultimately reported around 1-in 1,000. The exact numbers from memory may be off, but the point is as systems get highly complex, increased interfaces and interactions lead to more failure modes. Understanding them all is really tough
- blunte 8y agoI think a fundamental concept that is relatively ignored in press (mainstream or non-fringe at least) is in business priorities. As the essay points out, Boeing made decisions based on market (financial) factors all along the way. Of course they did - almost all companies do. Because it's so common, we forget it's not a RULE that cost, shareholder value, profit, etc. must be the final judgement. Certainly any company that might prioritize safety (as in the case of a people-carrier) or some other non financial focus may not be as attractive to investors, in some cases - particularly where human life is concerned - maybe that's ok. I know this is a bit sensational of me to suggest, but Walmart could make an airplane. They have the funds to do so if they wanted (they would just buy a company; that's the quick way to get rolling). And if the Walmart airline charged half of what other airlines charged, you can be sure a whole lot of people would fly it. And sure, after enough flights, the fatalities per flight would become uncomfortable to most flyers. The point is, it's a very long game. If people are not willing to consider the long game, they are basically gambling. I fly a lot. I almost always choose my flights based on cost/convenience. I didn't previously avoid 737 MAXs. (I would now, but they're all grounded anyway.) That said, if I know an airline cuts every corner to lower the price and increase the profits, I will certainly choose the next more expensive airline that doesn't behave so poorly. In this case it's the manufacturer, and the consumer has much less choice where that is involved. But let's get back to the point. If Boeing were to fall behind Airbus in the 737/A320 race, would that really be such a terrible thing? Would the cost be human life, or might it be some stock price level? As an investor, do you really care about your shareholder value more than human life? I like to fly. It can actually be fun. And I really like to visit lots of places in the world, eat lots of awesome food, and experience different cultures. I don't want to die because of some shareholder value goal (fuck you). I will die, and maybe it's tomorrow, but it shouldn't be for a stupid reason unless I choose it (here, hold my beer.)
- bumby 8y agoI agree that people often seem to miss that this may be rooted in a business decision. The problem is it isn't a risk-informed decision. I would doubt Boeing was accurately able to assess the actual risk of MCAS causing a catastrophic failure or else the decision to rush to market wouldn't have happened. I think just as large of a problem is mis-aligned incentives. Management is almost assuredly not singly focus on the extreme long term. The are graded quarter-by-quarter, year-by-year. This pushes uncertain risks to the periphery in favor of short term profits. I'm worried that unless the incentive structure is re-evaluated (by jail terms as an example) management will continue to make these types of decisions because schedule and cost will remain king
- stevespang 8y agoEXECUTIVE SUMMARY: So Boeing produced a dynamically unstable airframe, the 737 Max. That is big strike No. 1. Boeing then tried to mask the 737’s dynamic instability with a software system. Big strike No. 2. Finally, the software relied on systems known for their propensity to fail (angle-of-attack indicators) and did not appear to include even rudimentary provisions to cross-check the outputs of the angle-of-attack sensor against other sensors, or even the other angle-of-attack sensor. Big strike No. 3.
- cmurf 8y agoAuthor suggests the 737 is fly by wire, by saying there's no direct feedback in the control stick forces as it relates to control surface forces, but are rather artificially presented by computer. That's simply not true. Even in the case of MCAS, the side effect of stabilizer nose down trim is to make elevator backpressure on the yoke more forceful. It is not a simulation. That's presumably its design goal is, and why it was so simple without any safeguards for overcorrection or a failed sensor. I also don't like the suggestion the plane is longitudinally unstable due to the engines. That is simply not consistent with FAR 25.173 (a). A central feature of this stability requirement is the plane will recover from a stall merely be releasing back pressure. Not all airplanes do this because not all airplanes are designed that way nor are they required to, but FAR 23 (normal category) and FAR 25 (air transport category) aircraft are required to exhibit this kind of stability as well as lateral static stability. Yet people keep saying the airplane isn't stable and MCAS makes it stable. You can't have a damn switch that instantly makes the airplane unairworthy, with a damn airworthiness directive that tells pilots to solve one problem (runaway trim, or MCAS upset) by making the airplane unairworthy. If MCAS is there to make the plane airworthy, turning off autotrim makes the plane unairworthy. That's why I don't buy any claim that MCAS is there to make the plane airworthy, until there's a preponderance of evidence from reputable sources that includes an explanation how in the world such a thing is not a violation of FAR 25. The sensible explanation is it's a stick force moderator, in order to ensure the FAA didn't require a type certification for this make/model derivative. A plane with a type certificate triggers a requirement in FAR 61 for the pilot to obtain a type rating to fly planes with that particular type certification. I do find the suggestion of conspiracy plausible, among Boeing, the FAA, and airlines, to avoid 737 MAX type certification different from prior 737s. Consistent with that, is this week's ass covering by an FAA board saying they see no reason for additional 737 simulator training for pilots to fly 737 MAX, i.e. paving the way for a software update only solution for the current problem.
- DuskStar 8y agoThis article reads well. Unfortunately, it's filled with fundamental mistakes. > In the old days, when cables connected the pilot’s controls to the flying surfaces, you had to pull up, hard, if the airplane was trimmed to descend. You had to push, hard, if the airplane was trimmed to ascend. With computer oversight there is a loss of natural sense in the controls. In the 737 Max, there is no real “natural feel.” Whoops. The 737 is one of the few airliners produced today that DOES still directly connect the pilot's controls to flying surfaces. There's literally 12mm wires connecting the yoke to the control surfaces. So, in fact, the 737 does have "natural feel". In fact, that's the whole problem MCAS was designed to solve - how to add force to the yoke when at high AOA in certain speed regions, in order to ensure a linear AOA response. (Certification requires something along the lines of "5 pounds force -> 5 degree AOA, 10 pounds force -> 10 degree AOA, etc") If the 737 was fly-by-wire, this force could be added directly in software. Instead, they added this force by changing the aircraft's trim. > In a pinch, a human pilot could just look out the windshield to confirm visually and directly that, no, the aircraft is not pitched up dangerously. No, you can't. There isn't a visual indication for AOA - the author is confusing this with pitch. You can have a 45 degree AOA while still having the nose of the plane pointed at the horizon. (The author mentions using the artificial horizon for this purpose, too) If you could read AOA from sensors that didn't require being exposed to the airflow, planes would use them. I'm sure there's more, and it greatly reduces my confidence in the article.
- ncmncm 8y agoYes, it is unfortunate that these errors detracted from the thrust of the article.
- lisper 8y ago> There isn't a visual indication for AOA Private pilot here. What you say is, strictly speaking, true but that doesn't mean that you can't tell an awful lot about what's going on by looking out the window, and, more importantly, at the other flight instruments. If your attitude, airspeed, and rate of climb are all looking normal, then if the AOA says you're stalling it's almost certainly wrong. And it it says that your AOA is 70 degrees (which the Ethiopian airlines AOA sensor did) then it is definitely wrong. MCAS was designed to blindly trust a single AOA sensor, which was known to be prone to failures. To call that inexcusable would be quite the understatement.
- raz32dust 8y agoCould someone please ELI5 what exactly went wrong that caused the crash? I read the article but I still don't understand if the crash was caused by (a) The MCAS doing something it wasn't programmed to do (by specification) (b) MCAS was working as expected, but the expectations/assumptions were wrong (excluding pilot mistakes). (c) MCAS worked as designed, and the design was correct IF the pilots behaved like Boeing expected them to. Or has this question not been answered yet? I am talking purely from the software correctness point of view, irrespective of the fact that using software to work around this problem was already bad design.
- ncmncm 8y agoIt is none of the above. The MCAS worked as specified. The specification was criminally stupid. It violated at least two bedrock principles of avionics design. It would not have been approved at all if Boeing had not bent over backwards to draw attention away from it, for fear that pilots might have needed extra training, or the plane need more exanination.
- xigency 8y agoThe answer is B.
- tgsovlerkhgsel 8y ago"All of the above" The MCAS itself did exactly what it was programmed to do. One of the sensors feeding it, which was not redundant (!), malfunctioned, and MCAS acted on the data - garbage in, garbage out. By not having sufficient redundancy, MCAS was designed in a failure-prone way. This was justified because the failure would be like another kind of known failure mode (runaway trim), and pilots already had a checklist for it (the key point being to turn off electric trim). The problems with that are: 1. The problem a malfunctioning MCAS (acting "correctly" based on bad data) introduces is intermittent/erratic, making it harder to correctly react to it. Especially since the pilots weren't aware that a system that would act like this was on board! 2. Humans make mistakes in high-stress situations, which is why accepting something that introduces additional problems and relies on humans to handle them is a bad idea. 3. Turning off the electric trim also made it hard for the pilots to correct the situation, as it disables the manually-controlled but motor-powered trim. One theory is that the forces may have been too great for the backup solution (trimming manually) to work, a situation that old (but not current) manuals addressed. MCAS has a limit how much it will trim, but any trim input by the pilot resets it, and lets it move the trim by the limit amount again. This is just a layman's interpretation: From the flight data recorder, it seems like the Ethiopian ET302 pilots only partially corrected the wrong trim, then did turn trim off. After this, there was no noticeable adjustment in trim until the pilots turned electric trim back on to be able to use it. They used it to trim a little bit, and 5 seconds after their last trim input, MCAS triggered again and drove them into the ground. [1] On the other hand, the Lion Air JT610 crew doesn't seem to have turned trim off, just corrected it each time MCAS trimmed down (only for MCAS to trim back down 5 seconds later) [2] - until they once only trimmed a little against it, so that they hadn't corrected what the previous MCAS trim had done, but unlocked it for another one. MCAS applied more trim, and that was it. This could likely have been avoided by telling the pilots about MCAS and how it works, having a separate switch that kills MCAS but not electric trim, some sort of indication that MCAS is working, some sort of indication that the AoA sensor is broken, redundant sensors, and countless other things. The most likely reason why those weren't implemented seems (for me) that adding them would likely trigger the certification/training requirements that Boeing was trying to avoid. [1] https://leehamnews.com/2019/04/05/bjorns-corner-et302-crash-report-the-first-analysis/ https://leehamnews.com/2019/04/05/bjorns-corner-et302-crash-... [2] https://static.seattletimes.com/wp-content/uploads/2018/11/LionAir-BlackBox-WEB-1020x680.jpg https://static.seattletimes.com/wp-content/uploads/2018/11/L...
- euske 8y agoSlightly off-topic, but I generally found IEEE Spectrum having more interesting articles than Communications of the ACM. Maybe they're more inclined to the industrial side whereas CACM is more academic?
- outworlder 8y ago> When MCAS senses that the angle of attack is too high, it commands the aircraft’s trim system (the system that makes the plane go up or down) to lower the nose. It also does something else: It pushes the pilot’s control columns (the things the pilots pull or push on to raise or lower the aircraft’s nose) downward. Wait a minute. Since when does MCAS include a stick pusher? I haven't flown anything outside simulators yet, but AFAIK 737's do not have stick pushers of any kind.
- ncmncm 8y agoThis is an error in the article. 737-MAX is not a fly-by-wire system. The forces felt by pilots are derived directly from aerodynamic forces on the control surfaces.
- Xcelerate 8y agoJust a meta-comment, but I've noticed that for every single article posted about the 737 Max on HN, the top comments all tend to say that something in the linked article is dramatically wrong. I'm not sure what this means, but I find it interesting.
- mncharity 8y ago> I'm not sure what this means Medicine is struggling to adopt modern best practices for quality processes from other industries, including aviation. Journalism hasn't yet even noticed they're needed?
- cjbprime 8y agoI think the takeaway is that it's not usually the case that something this technical makes it to world news. Everyone wants to have an opinion, but even in this article's case of someone who's both a pilot and programmer, they apparently know next to nothing about how a 737 MAX is controlled (mechanically, not with forces artificially applied to inputs) because they fly a Cessna. And there aren't that many 737 pilots with the time or ability to write something frank about this without perhaps getting in trouble with their employers.
- PaulAJ 8y agoOne issue about the "bitey dog" MCAS not mentioned was the story of Air France 447 (https://en.wikipedia.org/wiki/Air_France_Flight_447 https://en.wikipedia.org/wiki/Air_France_Flight_447). Thats the one that stalled into the Atlantic in 2009. The flight crew plus a pilot that was riding in the back seat spent minutes trying to debug the flight computer, and only realised too late that the copilot was pulling back on the stick the whole time, keeping the aircraft stalled. This happened because of mode confusion: the aircraft computer realised it had compromised sensors and had switched to "alternate law", in which the computer would not override a stall. I have no doubt that Boeing knew about this incident and did not want to create a repeat.
- fogetti 8y agoLet me tell one thing: I absolutely hate sloppy engineering. And by sloppy I also mean rushing. There is a class of engineers and engineering managers who think that they are rockstars because they can churn out code quickly. I am absolutely disgusted and sick of this attitude. And why do I bring it up here? Because I suspect this kind of attitude was partly blamed for this fatal and tragic accident. Of course this is pure speculation. But my 12 years experience in the software industry makes me believe that's what happened. Also a relevant talk by Uncle Bob: https://youtu.be/ecIWPzGEbFc https://youtu.be/ecIWPzGEbFc
- robertAngst 8y ago>I absolutely hate sloppy engineering. >Of course this is pure speculation Cool ideas, people have already figured out this was designed into the system. Sure more testing can catch things, but how much money are you allowed to spend on every single feature? There isn't a 'right' answer to these questions, engineers are literally doing cutting edge, never before jobs. Everyone wants more time, more money, and better suppliers. You need a call to action, not just a youtube video talking about groups not being responsible.
- siwatanejo 8y agoCompletely agree.
- torgian 8y agoBut did they make it mobile first? Apparantly not.
- tlc1970 8y agoThis article increases my concern that the primary issue here is a failure in process to ensure the safety of passengers before the 737 Max went to market. We have world class expertise in the aviation industry, and there is no excuse for rushing a product to market without fully vetting it for safety. Boeing, the FAA, and the airlines moved too quickly to place the 737 Max in the air and too slowly to ground the planes after the second crash. In both cases it appears that these decisions were influenced by an over-reliance on the capabilities of technology over people.
- torgian 8y ago“ I believe the relative ease—not to mention the lack of tangible cost—of software updates has created a cultural laziness within the software engineering community. Moreover, because more and more of the hardware that we create is monitored and controlled by software, that cultural laziness is now creeping into hardware engineering—like building airliners. Less thought is now given to getting a design correct and simple up front because it’s so easy to fix what you didn’t get right later.” This is why I feel the software industry has too many problems with security, performance, etc. I’d like to think that engineers care how good and efficient their code is. But too often, it’s up to managers or customers how quickly software needs to be completed. This introduces bugs, incomplete features, and (in the case of Very Important Thigs to keep you Alive) potentially cause dangerous breaks. It sounds like Big Corp was trying to push that laziness and lack of foresight into other engineering disciplines. If so, how many of our cars, planes, and other items that potentially directly affect our lives are affected by mechanical design flaws and software errors? Hopefully the industry gets its shit together as a whole. As it stands, if I ever work on anything that affects a life, I’m damn well blowing a whistle if I feel like something is off.
- zerogvt 8y ago"So the FAA said to the airplane manufacturers, “Why don’t you just have your people tell us if your designs are safe?”" This
- tlc1970 8y agoFrankly, as a traveler, I have lost trust in Boeing, the FAA, and the airlines. This article underscores my concerns that the 737 Max was rushed to the market too quickly, and that there was an unacceptable delay in grounding these planes after two crashes. The expertise of the airline and software industries is world class. However, the process by which a determination was made to consider the Max 737 safe has failed us as travelers.