3 ms·

This does make one wonder about an architecture that has 4 or more identical processors hooked up to the same inputs and two watchdogs watching them (and each o

This does make one wonder about an architecture that has 4 or more identical processors hooked up to the same inputs and two watchdogs watching them (and each other). If any processor starts disagreeing with the rest it is instantly reset. If any processor starts to draw too much or too little current it is reset. If either watchdog stops responding to the other it is reset. Keeping the working data synchronized would be the real trick. One could imagine all of these CPUs are hooked to a single bank of redundant and ECC stabilized memory, and all access goes through the watchdog processes which will only let through the ones that are in agreement. The end result could be a system that is lighter and faster than the traditional RAD hardened system simply because it's built on a smaller process. The downside is the enormous complexity of the watchdog systems, they would be very expensive to get right. Also, synchronization is one of the hardest problems in computer science. It's basically cache invalidation on steroids.

mlyle 6y ago
> This does make one wonder about an architecture that has 4 or more identical processors hooked up to the same inputs and two watchdogs watching them (and each other). If any processor starts disagreeing with the rest it is instantly reset. If any processor starts to draw too much or too little current it is reset. If either watchdog stops responding to the other it is reset. This is a common set of techniques in critical systems. The smallsat built by students that I mentored didn't have voting, but it had processors with moderately sophisticated watchdogs performing mutual-power-monitoring and simple hardware failsafes. Aerospace control systems often run in lockstep and have voting, etc. > Keeping the working data synchronized would be the real trick. One could imagine all of these CPUs are hooked to a single bank of redundant and ECC stabilized memory, and all access goes through the watchdog processes which will only let through the ones that are in agreement. Typically you make the memory redundant too, and just ensure that input and outputs are common and all code is deterministic. e.g. There's Tandem / HP NonStop which use these techniques. > The end result could be a system that is lighter and faster than the traditional RAD hardened system simply because it's built on a smaller process. Handling upsets by voting makes a lot of sense. But radiation can cause permanent damage of small geometry circuits. And even with fast mechanisms to crowbar power in the event of a latchup, you're not really sure that you'll save the day. It's a great way to handle things on the cheap for cutting edge payloads and research projects, though.
- mech422 6y ago
  >>Typically you make the memory redundant too, and just ensure that input and outputs are common and all code is deterministic. e.g. There's Tandem / HP NonStop which use these techniques. Tandem was software fault tolerant - the Stratus machines were fully hardware fault tolerant and ran code in parallel on seperate logical cpus to test for faults. Truly neat machines. Also, it was a PL/1 based machine - even the OS.
  - mlyle 6y ago
    Yes, you're right. Turns out this doesn't apply to all of NonStop. I was thinking of Tandem NonStop CLX which my brother worked on and was lockstepped.
mech422 6y ago
This is rather like how the old Stratus super mini's worked.. 12(?) Motorola 68K chips were wired into multiple 'logical' cpu's, and programs executed in parallel on each. If a discrepancy arose, majority won and 'wrong' logical cpu's were taken out of service. It would also automatically call Stratus's remote service, report the failure and order a replacement board :-)

mlyle 6y ago

> This does make one wonder about an architecture that has 4 or more identical processors hooked up to the same inputs and two watchdogs watching them (and each other). If any processor starts disagreeing with the rest it is instantly reset. If any processor starts to draw too much or too little current it is reset. If either watchdog stops responding to the other it is reset. This is a common set of techniques in critical systems. The smallsat built by students that I mentored didn't have voting, but it had processors with moderately sophisticated watchdogs performing mutual-power-monitoring and simple hardware failsafes. Aerospace control systems often run in lockstep and have voting, etc. > Keeping the working data synchronized would be the real trick. One could imagine all of these CPUs are hooked to a single bank of redundant and ECC stabilized memory, and all access goes through the watchdog processes which will only let through the ones that are in agreement. Typically you make the memory redundant too, and just ensure that input and outputs are common and all code is deterministic. e.g. There's Tandem / HP NonStop which use these techniques. > The end result could be a system that is lighter and faster than the traditional RAD hardened system simply because it's built on a smaller process. Handling upsets by voting makes a lot of sense. But radiation can cause permanent damage of small geometry circuits. And even with fast mechanisms to crowbar power in the event of a latchup, you're not really sure that you'll save the day. It's a great way to handle things on the cheap for cutting edge payloads and research projects, though.

mech422 6y ago
>>Typically you make the memory redundant too, and just ensure that input and outputs are common and all code is deterministic. e.g. There's Tandem / HP NonStop which use these techniques. Tandem was software fault tolerant - the Stratus machines were fully hardware fault tolerant and ran code in parallel on seperate logical cpus to test for faults. Truly neat machines. Also, it was a PL/1 based machine - even the OS.
- mlyle 6y ago
  Yes, you're right. Turns out this doesn't apply to all of NonStop. I was thinking of Tandem NonStop CLX which my brother worked on and was lockstepped.

mech422 6y ago

This is rather like how the old Stratus super mini's worked.. 12(?) Motorola 68K chips were wired into multiple 'logical' cpu's, and programs executed in parallel on each. If a discrepancy arose, majority won and 'wrong' logical cpu's were taken out of service. It would also automatically call Stratus's remote service, report the failure and order a replacement board :-)