4 ms·

Was looking through Opportunity's Wikipedia page, and found this cool comparison of the embedded systems used in Mars rovers: https://en.wikipedia.org/wiki/Com

Was looking through Opportunity's Wikipedia page, and found this cool comparison of the embedded systems used in Mars rovers: https://en.wikipedia.org/wiki/Comparison_of_embedded_computer_systems_on_board_the_Mars_rovers https://en.wikipedia.org/wiki/Comparison_of_embedded_compute... That lead me to the RAD6000 page, which was new to me: https://en.wikipedia.org/wiki/IBM_RAD6000 https://en.wikipedia.org/wiki/IBM_RAD6000 "Reported to have a unit cost somewhere between US$200,000 and US$300,000, RAD6000 computers were released for sale in the general commercial market in 1996" Anybody know why the per unit cost is so high? Low yields or is it that much more expensive?

monocasa 8y ago
Low sales, meaning the R&D cost (which'll be higher than a non rad hard chip anyway) can't be as easily amortized. And probably half of it is or so is just the .gov markup.
deleted 8y ago
[deleted]
p1necone 8y ago
I would assume the kinds of use cases that people might need radiation hardened equipment for just have people willing to spend much more money. They're charging what people are willing to pay, not how much it costs to make.
kayfox 8y ago
Low yields, low volume, high development cost, high certification costs, strict QA. Vorago makes a cheaper option these days at $800 a chip: https://www.voragotech.com/products/va10820-radiation-hardened-arm®-cortex®-m0-mcu https://www.voragotech.com/products/va10820-radiation-harden...
- justin66 8y ago
  They're not comparable chips. The RAD6000 has a thousand times more RAM, for starters.
- snerbles 8y ago
  That's a Harvard architecture microcontroller, a bit different from the POWER CPUs in the rovers. Still subject to the same engineering constraints, but significantly less complex.
sitharus 8y ago
The costs of employing a team to do the radiation hardening design work, running a non-mainstream fab (I believe these chips were silicon-on-sapphire) and small production runs. There's a lot of overheads amortised on to a very small number of chips.
- Rebelgecko 8y ago
  I believe they're fabbed in Virginia, which probably doesn't help costs either
deleted 8y ago
[deleted]
civilitty 8y ago
My understanding is that most of that is setup costs for retooling a fab, which requires moving around and recalibrating some very specialized equipment. The numbers I heard were dozens of engineers working a total of one to two thousand man hours to set up the process, make and verify the chips, then switch the fab back to the processes their other customers usually order. Since these semiconductor fabs have high upfront and fixed operating costs, they need to have a high utilization to be profitable - preventing anyone from having a specialized fab that makes only RAD hardened chips. AFAIK the lower cost VORAGO designs require far less retooling so they're a lot cheaper with existing processes.
GlenTheMachine 8y ago
A couple of factors. 1) The yield rates for spaceflight-qualified chips is very, very low. Like 1%-5% or so. The chips are inspected when they come out of fab, and only the most perfect ones are given a spaceflight certification. The rest of the chips are used for other, less stringent applications (test boards, or military/embedded applications). 2) Spaceflight parts have significant paper trails. For metal parts, they are traced from the moment a lot of material comes out of the mill, and every time it is touched or changes hands thereafter that fact is recorded. Same thing with chips. Every chip has a "traveler" associated with it that records when it was manufactured, how it was stored, etc. Keeping those records costs a surprising amount of money. Handling the parts so the paper trail can be kept costs even more. You have to organize your logistics train such that every part is individually trackable. That reduces efficiency and adds cost. 3) Relatively low economies of scale.
- hinkley 8y ago
  Commercial aircraft also track provenance very carefully. I wonder how much the two systems have informed each other over the years. You would never want a part involved in a stress test to be reused, and you certainly don't want test parts anywhere near a production craft. This isn't like IT where you can cannibalize parts from a QA box to put into a production server.
  - rocketperson7 8y ago
    Having worked on mission critical commercial aircraft electronics, I can assure you every part is stress tested before it goes on a plane. Both after its first made and every time its in for service. Supposedly there has been research done showing that stress testing does not adversely affect reliability, at least at this quality level Also, the paper trail is quite literally paper. Its kinda amazing how slow the aviation industry is to adopt new things
    - hinkley 8y ago
      It's possible I'm mucking up the jargon, but isn't stress testing electronics what most of us might call "burn-in"? I definitely want the flight control computer burnt in before I sit on the airplane. I was thinking more of physical components, like an aluminum bracket, and you don't want a part with metal fatigue being installed as new.
luizfzs 8y ago
I saw a video once saying that SpaceX uses general purpose computers on the rockets instead of specific purpose hardware. If I'm not mistaken, they amount to 6 and have a checking system to assure the output of them is the same.
- planteen 8y ago
  Yes and a SpaceX rocket computer is not in space very long and is mostly in the low radiation environment of LEO. These Mars rovers spent years in a high radiation environment and received a large total ionizing dose of radiation. Having multiple computers doesn't do anything for total ionizing dose, they are all going to fail around the same time. I'm not sure about the RAD6000 being discussed here, but its successor, the RAD750, is fabbed with silicon-on-sapphire to help with total ionizing dose. For single event upsets, there is triple modular redundancy for all logic in the CPU.
- tzs 8y ago
  Those "have N copies and compare them" systems then have the issue that whatever does the comparison is a single point of failure. You could use multiple comparison units and have a second level comparison check them, but then that is your single point of failure, and so on. I've been told, but never actually looked it up, that there is a theorem that proves you always have to have at least one single point of failure. I don't know if the following is actually true, or just a rumor, but I've heard that at least one aircraft whose mission called for very high reliability didn't have a comparison unit: the redundancy extended all the way to having the 3 independent flight control computers each control a separate actuator on each flight control surface. If one of the systems went bad and tried to move the surface incorrectly, the other two would physically overpower it. That still has a single point of failure, but now that point is the control surface itself. If your control surface itself has failed it no longer matters if the 3 computers controlling it agree.
  - mehrdadn 8y ago
    This is something I'd always wondered about and never gotten a straight answer to. You're the first person I've seen who explicitly confirmed my suspicion that every system has a single point of failure. Would love to find a reference for this...

monocasa 8y ago

Low sales, meaning the R&D cost (which'll be higher than a non rad hard chip anyway) can't be as easily amortized. And probably half of it is or so is just the .gov markup.

deleted 8y ago

[deleted]

p1necone 8y ago

I would assume the kinds of use cases that people might need radiation hardened equipment for just have people willing to spend much more money. They're charging what people are willing to pay, not how much it costs to make.

kayfox 8y ago

Low yields, low volume, high development cost, high certification costs, strict QA. Vorago makes a cheaper option these days at $800 a chip: https://www.voragotech.com/products/va10820-radiation-hardened-arm®-cortex®-m0-mcu https://www.voragotech.com/products/va10820-radiation-harden...

justin66 8y ago
They're not comparable chips. The RAD6000 has a thousand times more RAM, for starters.
snerbles 8y ago
That's a Harvard architecture microcontroller, a bit different from the POWER CPUs in the rovers. Still subject to the same engineering constraints, but significantly less complex.

sitharus 8y ago

The costs of employing a team to do the radiation hardening design work, running a non-mainstream fab (I believe these chips were silicon-on-sapphire) and small production runs. There's a lot of overheads amortised on to a very small number of chips.

Rebelgecko 8y ago
I believe they're fabbed in Virginia, which probably doesn't help costs either

deleted 8y ago

[deleted]

civilitty 8y ago

My understanding is that most of that is setup costs for retooling a fab, which requires moving around and recalibrating some very specialized equipment. The numbers I heard were dozens of engineers working a total of one to two thousand man hours to set up the process, make and verify the chips, then switch the fab back to the processes their other customers usually order. Since these semiconductor fabs have high upfront and fixed operating costs, they need to have a high utilization to be profitable - preventing anyone from having a specialized fab that makes only RAD hardened chips. AFAIK the lower cost VORAGO designs require far less retooling so they're a lot cheaper with existing processes.

GlenTheMachine 8y ago

A couple of factors. 1) The yield rates for spaceflight-qualified chips is very, very low. Like 1%-5% or so. The chips are inspected when they come out of fab, and only the most perfect ones are given a spaceflight certification. The rest of the chips are used for other, less stringent applications (test boards, or military/embedded applications). 2) Spaceflight parts have significant paper trails. For metal parts, they are traced from the moment a lot of material comes out of the mill, and every time it is touched or changes hands thereafter that fact is recorded. Same thing with chips. Every chip has a "traveler" associated with it that records when it was manufactured, how it was stored, etc. Keeping those records costs a surprising amount of money. Handling the parts so the paper trail can be kept costs even more. You have to organize your logistics train such that every part is individually trackable. That reduces efficiency and adds cost. 3) Relatively low economies of scale.

hinkley 8y ago
Commercial aircraft also track provenance very carefully. I wonder how much the two systems have informed each other over the years. You would never want a part involved in a stress test to be reused, and you certainly don't want test parts anywhere near a production craft. This isn't like IT where you can cannibalize parts from a QA box to put into a production server.
- rocketperson7 8y ago
  Having worked on mission critical commercial aircraft electronics, I can assure you every part is stress tested before it goes on a plane. Both after its first made and every time its in for service. Supposedly there has been research done showing that stress testing does not adversely affect reliability, at least at this quality level Also, the paper trail is quite literally paper. Its kinda amazing how slow the aviation industry is to adopt new things
  - hinkley 8y ago
    It's possible I'm mucking up the jargon, but isn't stress testing electronics what most of us might call "burn-in"? I definitely want the flight control computer burnt in before I sit on the airplane. I was thinking more of physical components, like an aluminum bracket, and you don't want a part with metal fatigue being installed as new.

luizfzs 8y ago

I saw a video once saying that SpaceX uses general purpose computers on the rockets instead of specific purpose hardware. If I'm not mistaken, they amount to 6 and have a checking system to assure the output of them is the same.

planteen 8y ago
Yes and a SpaceX rocket computer is not in space very long and is mostly in the low radiation environment of LEO. These Mars rovers spent years in a high radiation environment and received a large total ionizing dose of radiation. Having multiple computers doesn't do anything for total ionizing dose, they are all going to fail around the same time. I'm not sure about the RAD6000 being discussed here, but its successor, the RAD750, is fabbed with silicon-on-sapphire to help with total ionizing dose. For single event upsets, there is triple modular redundancy for all logic in the CPU.
tzs 8y ago
Those "have N copies and compare them" systems then have the issue that whatever does the comparison is a single point of failure. You could use multiple comparison units and have a second level comparison check them, but then that is your single point of failure, and so on. I've been told, but never actually looked it up, that there is a theorem that proves you always have to have at least one single point of failure. I don't know if the following is actually true, or just a rumor, but I've heard that at least one aircraft whose mission called for very high reliability didn't have a comparison unit: the redundancy extended all the way to having the 3 independent flight control computers each control a separate actuator on each flight control surface. If one of the systems went bad and tried to move the surface incorrectly, the other two would physically overpower it. That still has a single point of failure, but now that point is the control surface itself. If your control surface itself has failed it no longer matters if the 3 computers controlling it agree.
- mehrdadn 8y ago
  This is something I'd always wondered about and never gotten a straight answer to. You're the first person I've seen who explicitly confirmed my suspicion that every system has a single point of failure. Would love to find a reference for this...