14 ms·

It's been a while since I read the detailed analysis, but wasn't it basically this? 1. A bug in the software that was prevented from doing harm (not necessaril

It's been a while since I read the detailed analysis, but wasn't it basically this? 1. A bug in the software that was prevented from doing harm (not necessarily masked but reduced to a minor irritant) by a hardware safety interlock. 2. The hardware safety interlock being removed in a cost-reduction. I know that "field mileage" is a powerful indicator of quality. Even if the code is awful and full of workarounds - if you have a lot of field experience you'd be loath to mess with it. So by the time the cost reduction came along the software would be fully trusted.

AlotOfReading 2y ago
Yes, that's essentially the story. It misses a lot of safeguards that should have existed (from a hindsight is 20/20 perspective). There's no "fully trusted" in safety critical code, so the software should have gone through a complete review when the safety model changed from hardware based to software based, including a HARA analysis and single point of failure analysis. The software should have had a specification and testing that included things like counter rollover and variable race conditions. The single developer should have had someone else to review their work and catch mistakes. This is standard practice these days, in large part specifically to prevent disasters like therac-25. While it's a little unfair to judge them by standards that didn't exist yet, it was still a broken process.
reginald78 2y ago
IIRC some of the underlying bugs came about because they used a much less dangerous device (X-Ray only) as the base and added the more dangerous system on top. The reused software components had different units of measure than the new mode which wasn't exposed until things were in "unexpected" error states. Then there was the usual arrogant gas lighting of anyone who reported the machine injuring and killing people. So the standard mix of computers are infallible and everyone in the change covering their asses by covering their ears and eyes so that the machine could kill again and again.

AlotOfReading 2y ago

Yes, that's essentially the story. It misses a lot of safeguards that should have existed (from a hindsight is 20/20 perspective). There's no "fully trusted" in safety critical code, so the software should have gone through a complete review when the safety model changed from hardware based to software based, including a HARA analysis and single point of failure analysis. The software should have had a specification and testing that included things like counter rollover and variable race conditions. The single developer should have had someone else to review their work and catch mistakes. This is standard practice these days, in large part specifically to prevent disasters like therac-25. While it's a little unfair to judge them by standards that didn't exist yet, it was still a broken process.

reginald78 2y ago

IIRC some of the underlying bugs came about because they used a much less dangerous device (X-Ray only) as the base and added the more dangerous system on top. The reused software components had different units of measure than the new mode which wasn't exposed until things were in "unexpected" error states. Then there was the usual arrogant gas lighting of anyone who reported the machine injuring and killing people. So the standard mix of computers are infallible and everyone in the change covering their asses by covering their ears and eyes so that the machine could kill again and again.