4 ms·

Shouldn't that rigorous testing regimen have preceded the sale and flight of these large passenger aircraft?

onychomys 7y ago
Sure. But I'd hope that they brought in new people, with a new outlook on things, and started all over from scratch. They're probably running mostly the same tests they were before the crashes, and also some new ones that people have dreamed up.
- tremon 7y ago
  I think it's more likely that all the managers in the chain of control suddenly feel like their ass is on the line if another 737 MAX drops out of the sky, which is why they're now paying more attention to what their engineers have been telling them for years. Which is probably a good thing, except for the part that it took hundreds of people to die first to get here.
kayfox 7y ago
A regime which finds all the defects would cost billions on its own, only one such piece of software has been subject to such a regime and that's the Space Shuttle guidance program. Every other piece of software is tested to levels set by the aeronautical industry and regulators, which may not include all possible scenarios. You will note, some of these scenarios are essentially fuzzing the memory of the flight computer and seeing what happens, ideally most bit flips will either be detected or be minor, but some can end up causing issues. I'm not sure if it would be possible to explore all the branches in the software in any sort of reasonable amount of time. https://en.wikipedia.org/wiki/Qantas_Flight_72 https://en.wikipedia.org/wiki/Qantas_Flight_72
- imglorp 7y ago
  That cost is already baked into aviation hardware. It's why fasteners are so expensive, for example, because the cost of certifying, tracking their history, and quantifying their quality is already amortized into each bolt. Flight software development lifecycle--and regs as you point out--still needs to catch up.
  - m4rtink 7y ago
    Actually, for fasteners there is often a material difference - cheaper fasteners are cut to shape while better fasteners are forged into shape, resulting in a much stronger part due to the resulting grain flow.
- andbberger 7y ago
  What's this about the space shuttle? I'm intrigued. I thought that the paris metro was the posterchild of formal verification methods.
  - salawat 7y ago
    The software that went into the Space Shuttle was coded to CMM level 5; this is a level of Quality Assurance that would make even the most Iron Fisted, detail oriented QA begin to sweat at the pile of paperwork and pressure ahead of them. It's a completely different experience that likely only a handful of modern testers would even contemplate nowadays, but has always been a shining example of something to aspire to be part of one day for me. It isn't at all easy, and basically requires the discipline to understand, document, and justify every single line of code within the context of the overarching system. No business wants to deal with it, and it is no surprise that only publically funded projects tend to get anywhere close. https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20110014946.pdf https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/201100... https://developers.slashdot.org/story/00/05/19/050258/space-shuttle-software-not-for-hacks https://developers.slashdot.org/story/00/05/19/050258/space-...
- tgflynn 7y ago
  > A regime which finds all the defects would cost billions on its own, only one such piece of software has been subject to such a regime and that's the Space Shuttle guidance program. Do you have a source for that claim ? I'm skeptical that even a testing process costing billions of dollars could claim to cover all possible inputs and states for a software system of any complexity.
  - raisedbyninjas 7y ago
    This describes not a testing regimen, but the entire development lifecycle. https://www.fastcompany.com/28121/they-write-right-stuff https://www.fastcompany.com/28121/they-write-right-stuff
    - tgflynn 7y ago
      It's a very interesting article but "as perfect as human beings have achieved" != perfect

Sure. But I'd hope that they brought in new people, with a new outlook on things, and started all over from scratch. They're probably running mostly the same tests they were before the crashes, and also some new ones that people have dreamed up.

tremon 7y ago
I think it's more likely that all the managers in the chain of control suddenly feel like their ass is on the line if another 737 MAX drops out of the sky, which is why they're now paying more attention to what their engineers have been telling them for years. Which is probably a good thing, except for the part that it took hundreds of people to die first to get here.

kayfox 7y ago

A regime which finds all the defects would cost billions on its own, only one such piece of software has been subject to such a regime and that's the Space Shuttle guidance program. Every other piece of software is tested to levels set by the aeronautical industry and regulators, which may not include all possible scenarios. You will note, some of these scenarios are essentially fuzzing the memory of the flight computer and seeing what happens, ideally most bit flips will either be detected or be minor, but some can end up causing issues. I'm not sure if it would be possible to explore all the branches in the software in any sort of reasonable amount of time. https://en.wikipedia.org/wiki/Qantas_Flight_72 https://en.wikipedia.org/wiki/Qantas_Flight_72

imglorp 7y ago
That cost is already baked into aviation hardware. It's why fasteners are so expensive, for example, because the cost of certifying, tracking their history, and quantifying their quality is already amortized into each bolt. Flight software development lifecycle--and regs as you point out--still needs to catch up.
- m4rtink 7y ago
  Actually, for fasteners there is often a material difference - cheaper fasteners are cut to shape while better fasteners are forged into shape, resulting in a much stronger part due to the resulting grain flow.
andbberger 7y ago
What's this about the space shuttle? I'm intrigued. I thought that the paris metro was the posterchild of formal verification methods.
- salawat 7y ago
  The software that went into the Space Shuttle was coded to CMM level 5; this is a level of Quality Assurance that would make even the most Iron Fisted, detail oriented QA begin to sweat at the pile of paperwork and pressure ahead of them. It's a completely different experience that likely only a handful of modern testers would even contemplate nowadays, but has always been a shining example of something to aspire to be part of one day for me. It isn't at all easy, and basically requires the discipline to understand, document, and justify every single line of code within the context of the overarching system. No business wants to deal with it, and it is no surprise that only publically funded projects tend to get anywhere close. https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20110014946.pdf https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/201100... https://developers.slashdot.org/story/00/05/19/050258/space-shuttle-software-not-for-hacks https://developers.slashdot.org/story/00/05/19/050258/space-...
tgflynn 7y ago
> A regime which finds all the defects would cost billions on its own, only one such piece of software has been subject to such a regime and that's the Space Shuttle guidance program. Do you have a source for that claim ? I'm skeptical that even a testing process costing billions of dollars could claim to cover all possible inputs and states for a software system of any complexity.
- raisedbyninjas 7y ago
  This describes not a testing regimen, but the entire development lifecycle. https://www.fastcompany.com/28121/they-write-right-stuff https://www.fastcompany.com/28121/they-write-right-stuff
  - tgflynn 7y ago
    It's a very interesting article but "as perfect as human beings have achieved" != perfect