6 ms·
> I've always seen this as an example of why physical/hardware interlocks are really important when you're mixing software with hardware that can easily hurt pe
by HeyLaughingBoy 7y ago
> I've always seen this as an example of why physical/hardware interlocks are really important when you're mixing software with hardware that can easily hurt people.
Agreed. I was a developer on a medical instrument where the decision to implement a hardware interlock was made only after we were years into development and months away from product release. Initially, the belief was that we could get away with software-monitored interlocks but then the lead HW engineer realized it would be an uphill battle to get UL to certify the machine as safe. UL & OSHA want to see "hard interlocks:" systems where opening an interlock immediately cuts power and eliminates the Hazard without anything else in the control chain.
The problem was that the easiest way for the hardware engineers to implement this: by cutting power to all the controllers, was also the hardest way to manage in software.
At any given moment, with dozens of motors and actuators processing commands, all I/O controllers could have their power cut when an operator opened the cover. This meant retrofitting graceful failure tolerance (previously our code expected that these events would mean a hard fault and a system shutdown) to every single I/O command, resulting in changes of thousands of lines of code, and months of work to implement and review the changes.
In the end it was the right thing to do: the machine was rendered safe as soon as an operator opened the cover while system software kept the non-affected parts of the machine running as best it could. As wolrah suggests in the sibling comment, this was an exceptional event: the only way to recover was to acknowledge the fault and put the instrument through a restart procedure.