4 ms·
"Over-reliance on unit testing"? I have a somewhat different read from the inquiry board report, which says it was a "systematic software design error." Yes, in
by dalke 10y ago
"Over-reliance on unit testing"? I have a somewhat different read from the inquiry board report, which says it was a "systematic software design error." Yes, inadequate testing, and a belief the working code for Ariane 4 meant it was validated for Ariane 5 were certainly contributory, but I think it's unfair to single out an over-reliance on unit testing.
The board argues that there was a bias towards believing the software does not have an error. Thus, any out-of-range value is interpreted as a hardware error, which means the CPU should shut down.
There was a decision to not include Ariane 5 trajectory data in the SRI requirements and specification. Thus, while tests were rigorous at the equipment ("unit") level, and there were system tests, they didn't test that case. This is test design failure.
In addition, the board says "the review process was a contributory factor in the failure."
I can see how those can be aspects of "over-reliance on unit testing", but it doesn't explain, for example, how some of the variables from Ariane 4 were protected from overflow exceptions but others were not.
- twic 10y agoYou have quite fairly picked me up on a cheeky exaggeration. Lots of things had to go wrong to cause the Ariane 5 failure - including bad handling of overflow, as you mention. But to my mind, the universal last line of defence against any kind of mistake is an integration test: put all of the parts of the system together, feed them real input, and verify that you get correct output. Arianespace did not do that. Well, until they actually launched it. It was a test flight, right? It proved to be an essential and very effective test.
- dalke 10y agoThat means there was an under-reliance on system testing, which I agree with. It doesn't imply there was an over-reliance on unit testing any more than it implies there was an over-reliance on analysis or over-reliance on the expectation of developing bug-free software.
- david-given 10y agoIf anyone's at all interested in risk analysis or robust software, it's totally worth finding a decent writeup on the Ariane 5 failure --- it's a fascinating postmortem in just how many unexpected little things together added up to a big boom. Everything is incredibly obvious in hindsight, of course, but making things obvious is largely what hindsight is for. And once you've finished reading that, go look up the Therac-25...