4 ms·

A regime which finds all the defects would cost billions on its own, only one such piece of software has been subject to such a regime and that's the Space Shut

by kayfox 7y ago

A regime which finds all the defects would cost billions on its own, only one such piece of software has been subject to such a regime and that's the Space Shuttle guidance program. Every other piece of software is tested to levels set by the aeronautical industry and regulators, which may not include all possible scenarios. You will note, some of these scenarios are essentially fuzzing the memory of the flight computer and seeing what happens, ideally most bit flips will either be detected or be minor, but some can end up causing issues. I'm not sure if it would be possible to explore all the branches in the software in any sort of reasonable amount of time. https://en.wikipedia.org/wiki/Qantas_Flight_72 https://en.wikipedia.org/wiki/Qantas_Flight_72

imglorp 7y ago
That cost is already baked into aviation hardware. It's why fasteners are so expensive, for example, because the cost of certifying, tracking their history, and quantifying their quality is already amortized into each bolt. Flight software development lifecycle--and regs as you point out--still needs to catch up.
- m4rtink 7y ago
  Actually, for fasteners there is often a material difference - cheaper fasteners are cut to shape while better fasteners are forged into shape, resulting in a much stronger part due to the resulting grain flow.
andbberger 7y ago
What's this about the space shuttle? I'm intrigued. I thought that the paris metro was the posterchild of formal verification methods.
- salawat 7y ago
  The software that went into the Space Shuttle was coded to CMM level 5; this is a level of Quality Assurance that would make even the most Iron Fisted, detail oriented QA begin to sweat at the pile of paperwork and pressure ahead of them. It's a completely different experience that likely only a handful of modern testers would even contemplate nowadays, but has always been a shining example of something to aspire to be part of one day for me. It isn't at all easy, and basically requires the discipline to understand, document, and justify every single line of code within the context of the overarching system. No business wants to deal with it, and it is no surprise that only publically funded projects tend to get anywhere close. https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20110014946.pdf https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/201100... https://developers.slashdot.org/story/00/05/19/050258/space-shuttle-software-not-for-hacks https://developers.slashdot.org/story/00/05/19/050258/space-...
tgflynn 7y ago
> A regime which finds all the defects would cost billions on its own, only one such piece of software has been subject to such a regime and that's the Space Shuttle guidance program. Do you have a source for that claim ? I'm skeptical that even a testing process costing billions of dollars could claim to cover all possible inputs and states for a software system of any complexity.
- raisedbyninjas 7y ago
  This describes not a testing regimen, but the entire development lifecycle. https://www.fastcompany.com/28121/they-write-right-stuff https://www.fastcompany.com/28121/they-write-right-stuff
  - tgflynn 7y ago
    It's a very interesting article but "as perfect as human beings have achieved" != perfect

by kayfox 7y ago

imglorp 7y ago

That cost is already baked into aviation hardware. It's why fasteners are so expensive, for example, because the cost of certifying, tracking their history, and quantifying their quality is already amortized into each bolt. Flight software development lifecycle--and regs as you point out--still needs to catch up.

m4rtink 7y ago
Actually, for fasteners there is often a material difference - cheaper fasteners are cut to shape while better fasteners are forged into shape, resulting in a much stronger part due to the resulting grain flow.

andbberger 7y ago

What's this about the space shuttle? I'm intrigued. I thought that the paris metro was the posterchild of formal verification methods.

salawat 7y ago
The software that went into the Space Shuttle was coded to CMM level 5; this is a level of Quality Assurance that would make even the most Iron Fisted, detail oriented QA begin to sweat at the pile of paperwork and pressure ahead of them. It's a completely different experience that likely only a handful of modern testers would even contemplate nowadays, but has always been a shining example of something to aspire to be part of one day for me. It isn't at all easy, and basically requires the discipline to understand, document, and justify every single line of code within the context of the overarching system. No business wants to deal with it, and it is no surprise that only publically funded projects tend to get anywhere close. https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20110014946.pdf https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/201100... https://developers.slashdot.org/story/00/05/19/050258/space-shuttle-software-not-for-hacks https://developers.slashdot.org/story/00/05/19/050258/space-...

tgflynn 7y ago

> A regime which finds all the defects would cost billions on its own, only one such piece of software has been subject to such a regime and that's the Space Shuttle guidance program. Do you have a source for that claim ? I'm skeptical that even a testing process costing billions of dollars could claim to cover all possible inputs and states for a software system of any complexity.

raisedbyninjas 7y ago
This describes not a testing regimen, but the entire development lifecycle. https://www.fastcompany.com/28121/they-write-right-stuff https://www.fastcompany.com/28121/they-write-right-stuff
- tgflynn 7y ago
  It's a very interesting article but "as perfect as human beings have achieved" != perfect