5 ms·
NASA Pulls Off 160-Million-Mile Software Patch
- SwaroopH 14y agoImpressive (even more for the geek me) but done before. Only relevant because of comparison with hellish OTA process for non-Google Android devices. https://twitter.com/#!/search/realtime/android%20nasa%20ota https://twitter.com/#!/search/realtime/android%20nasa%20ota
- noselasd 14y agoThey can even patch Voyager, now at the edge of our solar system: http://www.spaceflightnow.com/news/n1005/11voyager2/ http://www.spaceflightnow.com/news/n1005/11voyager2/ They also patched the MER rovers many times; This bug caused quite a few headlines back in the day: http://www.nasa.gov/offices/oce/llis/1483.html http://www.nasa.gov/offices/oce/llis/1483.html
- sehugg 14y agoI seem to remember a Scientific American article from the 1980s where they also had to rewrite code around a dead bit in RAM. They've done some pretty drastic changes since, like using the backup computer to add image compression on the fly (lots more detail here): http://history.nasa.gov/computers/Ch6-2.html http://history.nasa.gov/computers/Ch6-2.html
- sktrdie 14y agoI find it weird that the rover wouldn't have enough storage to hold both the logics of the landing and the surface mission. I mean, SSDs nowadays can hold hundreds of gigs and are very small. Does anybody have more details on that? A complete software update just sounds like a really risky thing, for something that you have no physical access to. My opinion is that they probably have really strict protocols from legacy missions, and since they just work, they're not going to change them.
- jgrahamc 14y agoEvery time there's a report about Curiosity doing something that overlaps with software or hardware that we at HN are familiar with comments like this appear. The bottom line is: (a) Curiosity was designed 8 years ago so the tech. on board is old (b) everything that's on board has to be radiation hardened and (c) NASA are conservative about 'new stuff' because they need it to work (e.g. the thrusters on the skycrane were derived from the Viking landers).
- papalalu 14y agopresumably the 8 year old design would use already-proven tech too..
- ballooney 14y agoIndeed, their shopping list 8 years ago was not from Newegg with whatever Intel had at the time, it was whatever BAE had spent many years rad hardening and qualifying by that point, which would itself have been based on an already proven CPU (read: older) when BAE first started looking. All of a sudden it's 1993.
- robryan 14y agoI wonder whether it will get to the point where they just send processors was a large amount of redundancy, multicore/ multiprocessor, and just not trust any one result but probabilistically across operations in parallel.
- harrywincup 14y agoIt must be unbearably frustrating building this stuff to work around the limitations of 10+ year-old technologies when the superior tech inside even the phone in your pocket could achieve so much more.
- mturmon 14y agoAlmost all of the radiation effects appear to be single event upsets (SEUs). So in principle, it should be possible to design robust software to detect and recover from faults (SEUs do no permanent damage to hardware). There have been efforts within NASA to design robust OS extensions -- redundancy, watchdogs, heartbeats, checkpoints, etc. -- so that fast/cheap COTS hardware could be used, at least for some functions like image processing. This has never reached critical mass and been carried to completion, however. So yes, as you guessed, there is significant pain and heartburn that computational capabilities are so far behind. Engineers have to spend a lot of time and cleverness squeezing, say, stereo vision processing, or image compression, into the hardware available.
- dmit 14y agoRelated Google Tech Talk: Debugging Code from 60 Million Miles Away https://www.youtube.com/watch?v=_gZK0tW8EhQ&hd=1 https://www.youtube.com/watch?v=_gZK0tW8EhQ&hd=1 Ron Garret talks about DS1, a project in the 90s where a small team used Lisp to send an autonomous craft into space in a third of the time and at a fraction of the cost of previous comparable projects. (Yes, I couldn't have made up a more HN-friendly story if I tried.)
- skrebbel 14y agoWait, they're doing this by hand? Production deployments to a computer on a different planet? In the 21st century? If there's one deploy I'd want to automate the hell of (and test, over a 2 baud network), it's this one. It's not like these guys never did a remote software update before. So I must be missing something.
- kbatten 14y agoWhat is your proposal to automate this? Remember there is very large delay and blackouts that happen. After they do it by hand several times, sure automate it.
- ehsanu1 14y agoAutomate it in a test environment that simulates production as closely as possible. Once the automated software updates work flawlessly under various conditions, use that in production.
- vidarh 14y agoThen an unexpected condition happens and the automated response breaks something badly, and suddenly your expensive robot is now an expensive piece of junk.
- jonknee 14y agoWhat makes you think they haven't already tested the hell out of it?
- rimantas 14y agoWe are not talking about some machine in amazon cloud there. We are talking about the robot on the freaking Mars.
- jonknee 14y agoWho said they're doing it by hand? It takes 28 minutes to receive feedback from anything, so they're going very slowly and monitoring what happens.
- sanarothe 14y agoI once nearly 'bricked' my router playing with some custom firmware. That in itself was terrifying, let alone the prospect of 'bricking the rover.'
- ahelwer 14y agoBeing responsible for the update would take years off of my life. One slip up and you've thrown away 2.5 billion dollars.
- sigkill 14y agoSure 2.5 billion dollars is a large sum of money. Imagine how you'd break this news to the public. You've practically crushed people's hopes and dreams. "Umm, guys, we just accidentally bricked the rover because of a firmware upgrade gone rogue". I'd end up in tears if I had done this. Speaking of firmwares, I'm unable to find out information if they have a dual bios kind of system where if the boot fails it can recover from the backup firmware.