3 ms·
Yes, we should go in that direction on anything safety critical. This is a real world situation that I encountered and led to me leaving the company, they did
by Jtsummers 10y ago
Yes, we should go in that direction on anything safety critical.
This is a real world situation that I encountered and led to me leaving the company, they did correct it after I left, however:
Fire detection system in multiple areas of a plane. If area A reported a fire before area B, then area B's report would be ignored. Reverse the situation and area A's report is ignored. This system, in particular, discovered fire/overheat early so that corrective action could be taken. By not making the second report it virtually guaranteed a fatal consequence should the (admittedly) very low probability event occur.
Also in this situation, like the aircraft in the article, there were a very small number of areas and sensors. Full, automated testing was entirely possible of literally every combination of sensor fault condition, ordering, and delays (to simulate "faulted" hardware, that is a sensor that reports a fire but isn't backed up by the other 2 sensors in its area). I calculated it at the time, the full series of automated tests could have been executed in a week, entirely reasonable given the lives we were responsible for.
However, we were not given a budget for this (until after I left), and so we were stuck with manual testing. Which drove the costs and time to an extreme amount (also made some testing effectively impossible, "flip this switch, flip this other switch after 3 but before 3.5 seconds"). The entire setup was a fucking joke. And lives depended on it.
On top of this, the software was a complete clusterfuck of shared variables used as temporaries across multiple subroutines, values getting erased (probably causing the problem that I discovered) when stepping through the piss-poor attempt at a state machine.
When hundreds of lives can be lost by the absence of a single, sensible, test, I'd gladly accept an extra couple million up front on a project, rather than deal with the emotional consequence of knowing my responsibility in their deaths, paying out millions to their widows and orphans, and possible civil and criminal consequences for the PEs involved.
- marcosdumay 10y agoThat's still depends entirely on the number of variables you must take into account. You are ok with an extra million of up-front cost, and that's reasonable. With a little bit of added complexity, that cost reaches an extra billion. Is it still ok? With a little more, and it's now a trillion. This is a really hard problem.
- Jtsummers 10y agoAnd that's where risk analysis comes in. But modeling the system, to a certain reasonable scale (read: feasible with respect to time), and sane programming practices can seriously mitigate the errors these systems have. I'd be willing to wager that the faulty system discussed in my quote had a set of variables like this at the top: int t1; int t2; int t3; Which are all used by various functions later on, obliterating the value the others set. And someone forgot that they were supposed to be temporary. I'd even put $100 on it (I'm poor, ok?).