5 ms·

This article repeats the claim that the MCAS software was faulty, but is there any evidence of coding-level errors? There is the systems engineering failure of

This article repeats the claim that the MCAS software was faulty, but is there any evidence of coding-level errors? There is the systems engineering failure of not comparing the two available AofA sensors to detect faults, if not going for at least triple-redundancy, but the root cause there is not a coding error. It has been suggested that high AofA readings should have been rejected unconditionally, but that approach can have its own problems, as in AF447 where the stall warnings stopped as the airplane became more deeply stalled. One should not leave the decision as whether to do that up to the programmer's discretion.

bdavis__ 7y ago
kind of depends on how you name it. my understanding is the code 'worked per requirements', meaning it did everything that it was supposed to, as defined by the requirements allocated to it. for things like this, there is a layer of 'systems engineering', that decides (hopefully using an engineering process) how the airplane will work. this is codified into a 'specification', that is then handed over to software to implement. if you are a software engineer given this task, you implement it per the specification. nothing more and nothing less. of course, if you have a concern, you bring it up (both informally and formally). there is no 'discretion' by the programmer. The number of AOA sensors, how much control authority, how often the inputs are applied, are all decided by systems engineers (with aeronautical engineers input and analysis). the software engineer cannot 'second guess' if AOA failure has been considered and change the implementation.
- _ph_ 7y ago
  The MCAS is a good example, why certain software must not be outsourced. Implementing the software correctly to spec isn't enough, you need enough understanding of the application to question the spec itself.
  - bdavis__ 7y ago
    anything that is connected to the flight controls, is too important to sub out (unless you are subbing it out to someone better that you are). making an airplane is a huge endeavor that crosses across many domains. you have to have competency up and down the 'stack'. Boeing does. Maybe this will cause a re-look at outsourcing policies. (edit. yes, this software was not outsourced. but it may add another dimension other than cost for future decisions)
    - prasooncc 7y ago
      It has been mentioned in the article that MCAS was never outsourced. Also it’s unfair to blame the software engineers who are building it to specification. It would be fantastical to assume that Boeing management( the same one ploughing through engineering with bean counting) would give a rats crap about a low level engineer raising a concern.
      - _ph_ 7y ago
        It has been mentioned in the article that MCAS was never outsourced. Where did the article say that? To me it explicitly claims it was outsourced: the McDonnell Douglas influenced Boeing new one tried to patch the problem with software. And it was bad software, written by to engineers paid $9/dollar an hour. And I am not blaming the software engineers for building it to the specification. In contrary, I clearly stated, that just building it to specification isn't enough for such critical software. Such software needs to be developed in teams where enough team members are senior enough to properly raise concerns where appropriate.
        glitchdout 7y ago
        Nowhere in the [original reporting][1] does it say that MCAS was outsourced. This article (The Coming Boeing Bailout?) wrongly asserts that MCAS was outsourced. There's no corroboration for that. Mind you, I'm not on Boeing's side. I'm just clarifying what's been reported so far. [1]: https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-737-max-software-outsourced-to-9-an-hour-engineers https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-...
  - partha42 7y ago
    Software engineers are not aerospace engineers. I doubt the software people would have understood the aerodynamic effects of the MCAS system by looking at the requirements.
    - bdavis__ 7y ago
      some would, some would not. those that have been doing aircraft software for 20 years probably have a fair knowledge of airplanes, aerodynamics, and flight controls. your comment expresses my feelings exactly on 'was this a software failure?'. No. But software was involved.
    - _ph_ 7y ago
      That is the problem with having pure software engineers implementing such a software. They need to have some aerospace engineering knowledge too. Or at least work directly in one team with aerospace engineers. I am working as a "software engineer" in the semiconductor industry. By education, I am a physicist. I am in my job for more than 10 years now, and beyond my university education, I have quite a grasp of electrical engineering. Furthermore, most of my colleagues are eletrical engineers. So in doubt, I can just walk over to their desk and discuss any question beyond pure software development issues. And some of the software developers here are just electrical engineers, which expanded their skill profile into software territory. Doing business this way is quite more expensive than having software developed by a contractor, but it creates safeguards.
      - sn41 7y ago
        That is true. But please keep in mind that the whole MCAS fiasco was precipitated by a dubious cost-cutting initiative related to an engine placed more in front than would be aerodynamically stable. The game was lost in the first quarter. The whole point is that civilian aircraft, unlike fighter jets, have to be aerodynamically stable without the need for such software, whether developed by holistic engineers, or compartmentalised subcontractors.
        _ph_ 7y ago
  - senderista 7y ago
    That assertion in the article seems to be incorrect according to the linked Bloomberg article, which states that the MCAS software was not outsourced: https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-737-max-software-outsourced-to-9-an-hour-engineers https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-.... Given that apparent factual error, I’m suspicious of the other claims, which are highly rhetorical and mostly asserted without evidence.
wsxcde 7y ago
> This article repeats the claim that the MCAS software was faulty, but is there any evidence of coding-level errors? No, there isn't. The Verge (and IIRC other sources) have reported that the crashes were due to failures in the angle of attack (AoA) sensors. No program in the world is going to give you correct output with garbage input. Verge link: https://www.theverge.com/2019/5/2/18518176/boeing-737-max-crash-problems-human-error-mcas-faa https://www.theverge.com/2019/5/2/18518176/boeing-737-max-cr...
- platz 7y ago
  The software did way more dumb things than it needed to in the presence of these failure modes.
  - mannykannot 7y ago
    Those 'dumb' things were, in fact, the things that the domain experts specified it should do in response to the data that the software received.
- michaelmrose 7y ago
  No sensor failure should cause the plan to go into swan dive to destruction mode. The software that carries your tweets has to deal with exceptional situations including hardware failure so to must software that runs life critical hardware. Seems to me that if the proper response to faulty sensors was to disable MCAS perhaps it should have just done that for the pilot. Even without a second AoA sensors perhaps something could have been derived from the fact that the pilot was furiously pulling up while the aircraft was losing altitude? Defense in depth is always a good thing.
  - DuskStar 7y ago
    > Even without a second AoA sensors perhaps something could have been derived from the fact that the pilot was furiously pulling up while the aircraft was losing altitude? People keep saying this. But AF 447 was caused by this. Pilot furiously pulling up, plane rapidly approaching the ground... And the steps to fix it would have been to push down. None of this is as simple as you seem to think.

bdavis__ 7y ago

kind of depends on how you name it. my understanding is the code 'worked per requirements', meaning it did everything that it was supposed to, as defined by the requirements allocated to it. for things like this, there is a layer of 'systems engineering', that decides (hopefully using an engineering process) how the airplane will work. this is codified into a 'specification', that is then handed over to software to implement. if you are a software engineer given this task, you implement it per the specification. nothing more and nothing less. of course, if you have a concern, you bring it up (both informally and formally). there is no 'discretion' by the programmer. The number of AOA sensors, how much control authority, how often the inputs are applied, are all decided by systems engineers (with aeronautical engineers input and analysis). the software engineer cannot 'second guess' if AOA failure has been considered and change the implementation.

_ph_ 7y ago
The MCAS is a good example, why certain software must not be outsourced. Implementing the software correctly to spec isn't enough, you need enough understanding of the application to question the spec itself.
- bdavis__ 7y ago
  anything that is connected to the flight controls, is too important to sub out (unless you are subbing it out to someone better that you are). making an airplane is a huge endeavor that crosses across many domains. you have to have competency up and down the 'stack'. Boeing does. Maybe this will cause a re-look at outsourcing policies. (edit. yes, this software was not outsourced. but it may add another dimension other than cost for future decisions)
  - prasooncc 7y ago
    It has been mentioned in the article that MCAS was never outsourced. Also it’s unfair to blame the software engineers who are building it to specification. It would be fantastical to assume that Boeing management( the same one ploughing through engineering with bean counting) would give a rats crap about a low level engineer raising a concern.
    - _ph_ 7y ago
      It has been mentioned in the article that MCAS was never outsourced. Where did the article say that? To me it explicitly claims it was outsourced: the McDonnell Douglas influenced Boeing new one tried to patch the problem with software. And it was bad software, written by to engineers paid $9/dollar an hour. And I am not blaming the software engineers for building it to the specification. In contrary, I clearly stated, that just building it to specification isn't enough for such critical software. Such software needs to be developed in teams where enough team members are senior enough to properly raise concerns where appropriate.
      - glitchdout 7y ago
        Nowhere in the [original reporting][1] does it say that MCAS was outsourced. This article (The Coming Boeing Bailout?) wrongly asserts that MCAS was outsourced. There's no corroboration for that. Mind you, I'm not on Boeing's side. I'm just clarifying what's been reported so far. [1]: https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-737-max-software-outsourced-to-9-an-hour-engineers https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-...
- partha42 7y ago
  Software engineers are not aerospace engineers. I doubt the software people would have understood the aerodynamic effects of the MCAS system by looking at the requirements.
  - bdavis__ 7y ago
    some would, some would not. those that have been doing aircraft software for 20 years probably have a fair knowledge of airplanes, aerodynamics, and flight controls. your comment expresses my feelings exactly on 'was this a software failure?'. No. But software was involved.
  - _ph_ 7y ago
    That is the problem with having pure software engineers implementing such a software. They need to have some aerospace engineering knowledge too. Or at least work directly in one team with aerospace engineers. I am working as a "software engineer" in the semiconductor industry. By education, I am a physicist. I am in my job for more than 10 years now, and beyond my university education, I have quite a grasp of electrical engineering. Furthermore, most of my colleagues are eletrical engineers. So in doubt, I can just walk over to their desk and discuss any question beyond pure software development issues. And some of the software developers here are just electrical engineers, which expanded their skill profile into software territory. Doing business this way is quite more expensive than having software developed by a contractor, but it creates safeguards.
    - sn41 7y ago
      That is true. But please keep in mind that the whole MCAS fiasco was precipitated by a dubious cost-cutting initiative related to an engine placed more in front than would be aerodynamically stable. The game was lost in the first quarter. The whole point is that civilian aircraft, unlike fighter jets, have to be aerodynamically stable without the need for such software, whether developed by holistic engineers, or compartmentalised subcontractors.
      - _ph_ 7y ago
- senderista 7y ago
  That assertion in the article seems to be incorrect according to the linked Bloomberg article, which states that the MCAS software was not outsourced: https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-737-max-software-outsourced-to-9-an-hour-engineers https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-.... Given that apparent factual error, I’m suspicious of the other claims, which are highly rhetorical and mostly asserted without evidence.

wsxcde 7y ago

> This article repeats the claim that the MCAS software was faulty, but is there any evidence of coding-level errors? No, there isn't. The Verge (and IIRC other sources) have reported that the crashes were due to failures in the angle of attack (AoA) sensors. No program in the world is going to give you correct output with garbage input. Verge link: https://www.theverge.com/2019/5/2/18518176/boeing-737-max-crash-problems-human-error-mcas-faa https://www.theverge.com/2019/5/2/18518176/boeing-737-max-cr...

platz 7y ago
The software did way more dumb things than it needed to in the presence of these failure modes.
- mannykannot 7y ago
  Those 'dumb' things were, in fact, the things that the domain experts specified it should do in response to the data that the software received.
michaelmrose 7y ago
No sensor failure should cause the plan to go into swan dive to destruction mode. The software that carries your tweets has to deal with exceptional situations including hardware failure so to must software that runs life critical hardware. Seems to me that if the proper response to faulty sensors was to disable MCAS perhaps it should have just done that for the pilot. Even without a second AoA sensors perhaps something could have been derived from the fact that the pilot was furiously pulling up while the aircraft was losing altitude? Defense in depth is always a good thing.
- DuskStar 7y ago
  > Even without a second AoA sensors perhaps something could have been derived from the fact that the pilot was furiously pulling up while the aircraft was losing altitude? People keep saying this. But AF 447 was caused by this. Pilot furiously pulling up, plane rapidly approaching the ground... And the steps to fix it would have been to push down. None of this is as simple as you seem to think.