6 ms·

Mars Curiosity Operating System: VxWorks

93 points by bradgessler 14y ago

samuel1604 14y ago
Glad to know it has support for IPv6, early adoption in new places is good!
- kokey 14y ago
  I thought they would just use the martian IPv4 ranges.
  - inopinatus 14y ago
    Very droll :) For the non-network engineers in the audience, it's worth documenting why this is funny: http://en.wikipedia.org/wiki/Martian_packet http://en.wikipedia.org/wiki/Martian_packet
    - noonespecial 14y ago
      "The name is derived from packet from Mars, a place where packets clearly can not originate." Uh oh. Looks like its time to update some wiki's.
- mcguire 14y ago
  "early adoption"?
skandl 14y ago
Previosly on Mars & VxWorks: http://www.cs.cmu.edu/afs/cs/user/raj/www/mars.html http://www.cs.cmu.edu/afs/cs/user/raj/www/mars.html
deleted 14y ago
[deleted]
davidw 14y ago
I wonder what the ground control software is written in. I know that for one of the previous ones, they used Tcl and Tk, and the first screenshots they showed with the first image were the X windowing system if I've ever seen it.
- zokier 14y ago
  I think I peered some SUN workstations, so Solaris would be a fair guess. Pure speculation of course.
  - yen223 14y ago
    How appropriate that a space agency will be using SUN workstations =)
losethos 14y ago
I wrote all 135,000 LOC in LoseThos, including its compiler. (God's laughing) God says... C:\Text\2CITIES.TXT sant sight too, beaming at all this in his little wig, and thanking his bachelor stars for having lighted him in his declining years to a Home. But, no Hundreds of people came to see the sights, and Mr. Lorry looked in vain for the fulfilment of Miss Pross's prediction. Dinner-time, and still no Hundreds of people. In the arrangements of the little household, Miss Pross took charge of the lower regions, and always acquitted herself marvellously. Her dinners, of a very modest quality, were so we
kokey 14y ago
I remember an HP tape array running VxWorks crashing when a copy of Nessus would scan it. There was no patch for the issue at that stage. This was a tape array backing up some infrastructure that would handle the transactions of a big chunk of Europe's stock exchanges. It kind of left me with a bad impression of VxWorks, but it might just be HP's processes that was a mess at that stage.
nikcub 14y ago
If anybody else is interested in the topic, the seminal work on software development at NASA is the Fast Company feature from 1996, "They write the right stuff": http://www.fastcompany.com/node/28121/print?donttaseme http://www.fastcompany.com/node/28121/print?donttaseme
- sigkill 14y ago
  I've read that over a dozen times by now and I'm still fascinated by it.
- smcl 14y ago
  Why the "donttaseme" in the query string?
  - qxcv 14y ago
    $ vimdiff <(curl http://www.fastcompany.com/node/28121/print?donttaseme http://www.fastcompany.com/node/28121/print?donttaseme) <(curl http://www.fastcompany.com/node/28121/print http://www.fastcompany.com/node/28121/print) Aesthetics, perhaps?
  - nikcub 14y ago
    I had it bookmarked like that, my guess is that it was a re-submission and somebody added that tag in order to make the URL unique
- fidotron 14y ago
  Worth saying that doesn't apply outside manned missions. They notoriously crashed a probe into Mars thanks to erroneous imperial to metric unit conversion. EDIT: also the VxWorks fan stuff here is a bit odd, given the other notorious error involved setting the task priorities so that the probe spent the whole time checking the weather, but never phoned home. They were just lucky they shipped the thing in debug mode and reprogrammed it once it had landed.
  - jrabone 14y ago
    That pathfinder bug was a priority inversion caused by failing to enable priority inheritance when creating mutexes, not quite as simple as setting the priorities wrong. VxWorks wasn't entirely at fault, although the priority inheritance feature was not obvious. Concurrency is hard. See http://research.microsoft.com/en-us/um/people/mbj/mars_pathfinder/mars_pathfinder.html http://research.microsoft.com/en-us/um/people/mbj/mars_pathf... for details.
xarien 14y ago
VxWorks is used pretty extensively in space and defense contracting. The list in wikipedia is a pretty small sublist. It's also often used as a wrapper for custom bootroms.
wyuenho 14y ago
Some features of VxWorks and why it's so deterministic and fast. http://stackoverflow.com/questions/8800371/what-makes-vxworks-so-deterministic-and-fast http://stackoverflow.com/questions/8800371/what-makes-vxwork...
joelthelion 14y ago
Do people still use VxWorks for new projects? Isn't linux gaining weight in the embedded market?
- jk4930 14y ago
  In space they increasingly use C++ on Linux (SpaceX, ESA, ...).
- yitchelle 14y ago
  Try getting some functional safety certification on a Linux distribution and you see why Linux is not really gaining much weight in certain areas of the embedded market. However, it is getting small inroads in the instrument clusters for Automobiles. (http://www.genivi.org/ http://www.genivi.org/)
- avar 14y ago
  Aside from what others have mentioned keep in mind that something being sent to another planet usually has 5-10 year old technology at best. For reference in 2003 Linux 2.4 was the latest stable version.
- jbester 14y ago
  Defense and aerospace - even there we are seeing more Linux for non-safety critical software
- gautamc 14y ago
  There was an article in linux-journal that described how linux was being used at JPL : http://www.linuxjournal.com/article/7570 http://www.linuxjournal.com/article/7570 From what I understand, Linux runs the rover simulator environment - known as RSVP. It consists of a DSL that is used to direct the rover's operations and a visualization environment. This way the ground crew can first plan and test the activities they want the rover to perform in this simulated environment and then upload the action plan to the rover - so overall Linux plays a pretty important part in the the rover tech stack. Most probably the initial rovers ran VxWorks RTOS, because back then it was the best choice (even now it isn't a bad choice at all) - complex systems are built incrementally.
- 14y ago
Two9A 14y ago
I was watching the Planetfest rebroadcast of Curiosity's landing, and mention was made of there needing to be two operating systems: one to deal with entry and landing, and the other to handle roving and research. The speaker went on to say that both OSes don't fit into RAM at once, so they would have to be swapped over post-landing, and that this would take four days to load from Flash. Raises the question: why does it take four days to swap an operating system in from Flash, and how does the system go about it while having a path to fall back?
- mbell 14y ago
  I'm guessing the copying doesn't take four days unless they are actually sending the program from earth, in which case ~256MB over X-band to Mars could take that long. Rather I would guess that this copy is a many step procedure they do one step at a time and confirm at every step. With a 28 minute round trip to tell the rover to do something and get a response time adds up quick.
- TazeTSchnitzel 14y ago
  I wonder if the time taken is because it must use very tiny amounts of power, and survive at such a low temperature. They may be using a very slow processor, or a special low-power, low-temperature NAND or whatever variant.
  - uxp 14y ago
    It depends on how one defines slow. Curiosity's onboard computer is 100x faster than the previous two Mars rovers, and has twice as much ram. But that's only 200MHz and 256MB respectively. With the exception of Sojourner, the first rover successfully placed on Mars, all Mars rovers have run VxWorks. http://en.wikipedia.org/wiki/Comparison_of_embedded_computer_systems_on_board_the_Mars_rovers http://en.wikipedia.org/wiki/Comparison_of_embedded_computer...
- yen223 14y ago
  Personal theory: The hardware used has to be low powered, and be able to withstand physical and radiological abuse. Such a system probably will not post the fastest clock cycles, what with power consumption being proportional to frequency^2 and all. I'm also going to guess that most of the available RAM will be used for redundancy.
sgt 14y ago
What does SpaceX use? Anyone got juicy details?
- basicxman 14y ago
  VxWorks as well, http://www.spacex.com/downloads/dragonlab-datasheet.pdf http://www.spacex.com/downloads/dragonlab-datasheet.pdf
- lavezza 14y ago
  From their careers page: "Our Flight Software Group currently has opportunities developing software for embedded flight hardware using Linux and VxWorks as well as ground simulation software using Linux."
fungi 14y ago
anyway to get my hands on it to have a play?
- jtdowney 14y ago
  Volunteer to mentor a FIRST Robotics Competition (http://www.usfirst.org/roboticsprograms/frc http://www.usfirst.org/roboticsprograms/frc) team. The robots built for the competition use a National Instrument cRIO (http://en.wikipedia.org/wiki/CRIO http://en.wikipedia.org/wiki/CRIO) which run VxWorks 6.3.
happywolf 14y ago
I worked on VxWorks and Linux on Intel's IXP embedded processors before, and VxWorks' performance and memory footprint were always better than Linux's. For 'Performance' I refer to network throughput because IXP was meant to be for routers and gateways. It was around 2006 so I am not sure what these two OSes are up to now. Don't flame me because I am just speaking from my own experience.
chucknelson 14y ago
Cool, now I know that Apple routers use the same OS as Curiosity.
- zeethrow 14y ago
  And Linksys.
akandiah 14y ago
SpaceX also use it for their Dragon spacecraft: http://www.spacex.com/downloads/dragonlab-datasheet.pdf http://www.spacex.com/downloads/dragonlab-datasheet.pdf
damian2000 14y ago
Pricey too - according to this from back in 2008, its $16,500 per seat (developer)... http://mihirknows.blogspot.com.au/2008/03/rtos-cost-and-feature-comparison.html http://mihirknows.blogspot.com.au/2008/03/rtos-cost-and-feat... A drop in the ocean in the scheme of space projects though.
huntall 14y ago
VxWorks operating system of Mars Curiosity http://huntall.com/vxworks-operating-system-of-mars-curiosity http://huntall.com/vxworks-operating-system-of-mars-curiosit...

93 points by bradgessler 14y ago

samuel1604 14y ago

Glad to know it has support for IPv6, early adoption in new places is good!

kokey 14y ago
I thought they would just use the martian IPv4 ranges.
- inopinatus 14y ago
  Very droll :) For the non-network engineers in the audience, it's worth documenting why this is funny: http://en.wikipedia.org/wiki/Martian_packet http://en.wikipedia.org/wiki/Martian_packet
  - noonespecial 14y ago
    "The name is derived from packet from Mars, a place where packets clearly can not originate." Uh oh. Looks like its time to update some wiki's.
mcguire 14y ago
"early adoption"?

skandl 14y ago

Previosly on Mars & VxWorks: http://www.cs.cmu.edu/afs/cs/user/raj/www/mars.html http://www.cs.cmu.edu/afs/cs/user/raj/www/mars.html

deleted 14y ago

[deleted]

davidw 14y ago

I wonder what the ground control software is written in. I know that for one of the previous ones, they used Tcl and Tk, and the first screenshots they showed with the first image were the X windowing system if I've ever seen it.

zokier 14y ago
I think I peered some SUN workstations, so Solaris would be a fair guess. Pure speculation of course.
- yen223 14y ago
  How appropriate that a space agency will be using SUN workstations =)

losethos 14y ago

I wrote all 135,000 LOC in LoseThos, including its compiler. (God's laughing) God says... C:\Text\2CITIES.TXT sant sight too, beaming at all this in his little wig, and thanking his bachelor stars for having lighted him in his declining years to a Home. But, no Hundreds of people came to see the sights, and Mr. Lorry looked in vain for the fulfilment of Miss Pross's prediction. Dinner-time, and still no Hundreds of people. In the arrangements of the little household, Miss Pross took charge of the lower regions, and always acquitted herself marvellously. Her dinners, of a very modest quality, were so we

kokey 14y ago

I remember an HP tape array running VxWorks crashing when a copy of Nessus would scan it. There was no patch for the issue at that stage. This was a tape array backing up some infrastructure that would handle the transactions of a big chunk of Europe's stock exchanges. It kind of left me with a bad impression of VxWorks, but it might just be HP's processes that was a mess at that stage.

nikcub 14y ago

If anybody else is interested in the topic, the seminal work on software development at NASA is the Fast Company feature from 1996, "They write the right stuff": http://www.fastcompany.com/node/28121/print?donttaseme http://www.fastcompany.com/node/28121/print?donttaseme

sigkill 14y ago
I've read that over a dozen times by now and I'm still fascinated by it.
smcl 14y ago
Why the "donttaseme" in the query string?
- qxcv 14y ago
  $ vimdiff <(curl http://www.fastcompany.com/node/28121/print?donttaseme http://www.fastcompany.com/node/28121/print?donttaseme) <(curl http://www.fastcompany.com/node/28121/print http://www.fastcompany.com/node/28121/print) Aesthetics, perhaps?
- nikcub 14y ago
  I had it bookmarked like that, my guess is that it was a re-submission and somebody added that tag in order to make the URL unique
fidotron 14y ago
Worth saying that doesn't apply outside manned missions. They notoriously crashed a probe into Mars thanks to erroneous imperial to metric unit conversion. EDIT: also the VxWorks fan stuff here is a bit odd, given the other notorious error involved setting the task priorities so that the probe spent the whole time checking the weather, but never phoned home. They were just lucky they shipped the thing in debug mode and reprogrammed it once it had landed.
- jrabone 14y ago
  That pathfinder bug was a priority inversion caused by failing to enable priority inheritance when creating mutexes, not quite as simple as setting the priorities wrong. VxWorks wasn't entirely at fault, although the priority inheritance feature was not obvious. Concurrency is hard. See http://research.microsoft.com/en-us/um/people/mbj/mars_pathfinder/mars_pathfinder.html http://research.microsoft.com/en-us/um/people/mbj/mars_pathf... for details.

xarien 14y ago

VxWorks is used pretty extensively in space and defense contracting. The list in wikipedia is a pretty small sublist. It's also often used as a wrapper for custom bootroms.

wyuenho 14y ago

Some features of VxWorks and why it's so deterministic and fast. http://stackoverflow.com/questions/8800371/what-makes-vxworks-so-deterministic-and-fast http://stackoverflow.com/questions/8800371/what-makes-vxwork...

joelthelion 14y ago

Do people still use VxWorks for new projects? Isn't linux gaining weight in the embedded market?

jk4930 14y ago
In space they increasingly use C++ on Linux (SpaceX, ESA, ...).
yitchelle 14y ago
Try getting some functional safety certification on a Linux distribution and you see why Linux is not really gaining much weight in certain areas of the embedded market. However, it is getting small inroads in the instrument clusters for Automobiles. (http://www.genivi.org/ http://www.genivi.org/)
avar 14y ago
Aside from what others have mentioned keep in mind that something being sent to another planet usually has 5-10 year old technology at best. For reference in 2003 Linux 2.4 was the latest stable version.
jbester 14y ago
Defense and aerospace - even there we are seeing more Linux for non-safety critical software
gautamc 14y ago
There was an article in linux-journal that described how linux was being used at JPL : http://www.linuxjournal.com/article/7570 http://www.linuxjournal.com/article/7570 From what I understand, Linux runs the rover simulator environment - known as RSVP. It consists of a DSL that is used to direct the rover's operations and a visualization environment. This way the ground crew can first plan and test the activities they want the rover to perform in this simulated environment and then upload the action plan to the rover - so overall Linux plays a pretty important part in the the rover tech stack. Most probably the initial rovers ran VxWorks RTOS, because back then it was the best choice (even now it isn't a bad choice at all) - complex systems are built incrementally.
14y ago

Two9A 14y ago

I was watching the Planetfest rebroadcast of Curiosity's landing, and mention was made of there needing to be two operating systems: one to deal with entry and landing, and the other to handle roving and research. The speaker went on to say that both OSes don't fit into RAM at once, so they would have to be swapped over post-landing, and that this would take four days to load from Flash. Raises the question: why does it take four days to swap an operating system in from Flash, and how does the system go about it while having a path to fall back?

mbell 14y ago
I'm guessing the copying doesn't take four days unless they are actually sending the program from earth, in which case ~256MB over X-band to Mars could take that long. Rather I would guess that this copy is a many step procedure they do one step at a time and confirm at every step. With a 28 minute round trip to tell the rover to do something and get a response time adds up quick.
TazeTSchnitzel 14y ago
I wonder if the time taken is because it must use very tiny amounts of power, and survive at such a low temperature. They may be using a very slow processor, or a special low-power, low-temperature NAND or whatever variant.
- uxp 14y ago
  It depends on how one defines slow. Curiosity's onboard computer is 100x faster than the previous two Mars rovers, and has twice as much ram. But that's only 200MHz and 256MB respectively. With the exception of Sojourner, the first rover successfully placed on Mars, all Mars rovers have run VxWorks. http://en.wikipedia.org/wiki/Comparison_of_embedded_computer_systems_on_board_the_Mars_rovers http://en.wikipedia.org/wiki/Comparison_of_embedded_computer...
yen223 14y ago
Personal theory: The hardware used has to be low powered, and be able to withstand physical and radiological abuse. Such a system probably will not post the fastest clock cycles, what with power consumption being proportional to frequency^2 and all. I'm also going to guess that most of the available RAM will be used for redundancy.

sgt 14y ago

What does SpaceX use? Anyone got juicy details?

basicxman 14y ago
VxWorks as well, http://www.spacex.com/downloads/dragonlab-datasheet.pdf http://www.spacex.com/downloads/dragonlab-datasheet.pdf
lavezza 14y ago
From their careers page: "Our Flight Software Group currently has opportunities developing software for embedded flight hardware using Linux and VxWorks as well as ground simulation software using Linux."

fungi 14y ago

anyway to get my hands on it to have a play?

jtdowney 14y ago
Volunteer to mentor a FIRST Robotics Competition (http://www.usfirst.org/roboticsprograms/frc http://www.usfirst.org/roboticsprograms/frc) team. The robots built for the competition use a National Instrument cRIO (http://en.wikipedia.org/wiki/CRIO http://en.wikipedia.org/wiki/CRIO) which run VxWorks 6.3.

happywolf 14y ago

I worked on VxWorks and Linux on Intel's IXP embedded processors before, and VxWorks' performance and memory footprint were always better than Linux's. For 'Performance' I refer to network throughput because IXP was meant to be for routers and gateways. It was around 2006 so I am not sure what these two OSes are up to now. Don't flame me because I am just speaking from my own experience.

chucknelson 14y ago

Cool, now I know that Apple routers use the same OS as Curiosity.

zeethrow 14y ago
And Linksys.

akandiah 14y ago

SpaceX also use it for their Dragon spacecraft: http://www.spacex.com/downloads/dragonlab-datasheet.pdf http://www.spacex.com/downloads/dragonlab-datasheet.pdf

damian2000 14y ago

Pricey too - according to this from back in 2008, its $16,500 per seat (developer)... http://mihirknows.blogspot.com.au/2008/03/rtos-cost-and-feature-comparison.html http://mihirknows.blogspot.com.au/2008/03/rtos-cost-and-feat... A drop in the ocean in the scheme of space projects though.

huntall 14y ago

VxWorks operating system of Mars Curiosity http://huntall.com/vxworks-operating-system-of-mars-curiosity http://huntall.com/vxworks-operating-system-of-mars-curiosit...