15 ms·

A Look into NASA’s Coding Philosophy

247 points by astdb 9y ago

bluedino 9y ago
I'd like to see how this compares to, SpaceX for example
- dsfyu404ed 9y ago
  When it comes to physical components SpaceX has a habit of selecting an off the shelf component and asking the vendor if they can get a modified version that includes some changes that make it capable of living in their application. That's great for reducing cost (or increasing quality or buying time, they're all related) but in the long term it often leads to spaghetti. Considering the time pressure they're perpetually under and how Elon is known to run his organizations I would suspect their software products are no different.
  - brianwawok 9y ago
    On that note, what language(s) are SpaceX, Tesla built on? Job postings I see a mix of C# and golang. I cannot believe either a Model X or a rocket is rockin C#...
    - 0xffff2 9y ago
      Probably not on the rocket itself, but C# is a very common language for ancillary testing tools in the industrial automation world. I wouldn't be surprised if a lot of their test infrastructure is written in C#.
      - w_t_payne 9y ago
        Pity. I've been using Python a lot for test infrastructure and it fits really well.
    - elteto 9y ago
      The software team did an AMA on reddit a while back: https://www.reddit.com/r/IAmA/comments/1853ap/we_are_spacex_software_engineers_we_launch/ https://www.reddit.com/r/IAmA/comments/1853ap/we_are_spacex_... Basically it's C/C++ (as expected) for critical stuff, like the rocket or dragon, and C# for all enterprise systems. Plus an assorted set of other languages/technologies depending on team and project.
- mcguire 9y ago
  The last I heard, from outside the loop, there was considerable concern about how they were going to get man rated.
lisper 9y ago
This article is very much at odds with my 15-year (1988-2000, 2001-2004) experience writing software at NASA (albeit at JPL, which is to say the unmanned space program, but still NASA). In my experience there was just as much politics, marketing hype, and general bullshit as I've seen in the commercial world. It was actually pretty amazing to me some times that things worked at all, let alone that they worked as well as they did. I think the real secret at NASA is not that they are better at writing code, but that they write it slowly (development cycles are measured in years) and they test the living shit out of it. Then, once it's working, they don't change it except under extreme pressure. But while it's being developed it's just as much of a sausage factory as anything else.
- sethrin 9y ago
  Does that mean that as long as you test the shit out of it, it doesn't matter how it gets done?
  - lifeisstillgood 9y ago
    That seems a reasonable position - yes.
    - pbhjpbhj 9y ago
      Reasonable if you ignore maintainability and standards compliance. Not a coder, but if you used an idiom that worked because of a compiler quirk then that could lead to problems down the line but related to maintenance per se, hence why I added the second part.
      - SomeHacker44 9y ago
        In any mission critical situation the entire tool chain is just as locked down and controlled as the "source code," even down to the silicon.
        olewhalehunter 9y ago
alexanderdmitri 9y ago
Here's a paper[1] for a space conference in Montreal written a few years ago on DO-178 standards and applying the FAA's certification process to space systems development (DO-178 sets the bar for "testing the shit" out of software, I've seen projects held up for years because it's like approaching a limit f(n) -> L (where f(n) is the software development cycle and L is getting a non-bought DER to sign off on it) proving your system to its requirements). [1] http://articles.adsabs.harvard.edu/cgi-bin/nph-iarticle_query?2013ESASP.715E..63D&data_type=PDF_HIGH&whole_paper=YES&type=PRINTER&filetype=.pdf http://articles.adsabs.harvard.edu/cgi-bin/nph-iarticle_quer...
- nickpsecurity 9y ago
  DO-178B was one of best things to happen in software assurance. It was a non-prescriptive method of certification that forced precise requirements, design, and code with reviews of each and tracability between them. The high cost of certification and certification failure led to a booming ecosystem of supporting tools (esp static analysis or runtimes) and pre-made components (esp RTOS's & middleware). Some companies even specialized in handling the administrative overhead. The result was that certified systems usually had very, high quality. Meanwhile, there were people arguing on HN and other places about whether a certification could ever be useful for software. Anyone reading the above paper should skip to the summary to get a high-level view of the overhead and resulting benefits. The author of the paper agrees with me that the certification is beneficial. The gist is you keep things simple, think about everything in lifecycle, test everything in lifecycle, throw every analysis tool you can at the design or code, esp take your time to develop the thing, and document all of this in a way where any aspect is easy to verify by a 3rd party. That's how high-assurance systems are done. They usually work, too. Some work for a long, long, long time. :)
  - mcguire 9y ago
    DO-178C has a formal methods follow-on. I haven't seen it, though.
  - ptero 9y ago
    Applied very narrowly to the right problem, maybe. But I have also seen DO-178B to lead to a completely non-maintainable mess. That mess helps trace each statement to a requirement, but leads to code that is unreadable to anyone used to normal code. It is not that "no extra code" and exactly meeting requirements is a bad model in itself. However writing code in a way that makes it convenient to prove the above encourages unnatural coding styles and programmatic contortions. YMMV however.
deleted 9y ago
[deleted]
Judgmentality 9y ago
I always found this article(1) about writing code for NASA fascinating. I'm sure a lot has changed in the 20 years since it was written, but it's such a stark contrast to what I'm used to. (1) https://www.fastcompany.com/28121/they-write-right-stuff https://www.fastcompany.com/28121/they-write-right-stuff
- mcguire 9y ago
  20 years ago: * 420,000 LOC * 260 people * $35,000,000 / year
  - Judgmentality 9y ago
    There's something to be said about doing more with less.
rav 9y ago
The article severely misrepresents Amdahl's law, in particular in the sentence: > Though you took half of your entire program, and optimized it to be four times faster, the overall system is only 1.6x faster. This application of Amdahl's law doesn't tell you anything about what happens when you take half of the program; it tells you what happens when you take half of the execution time. If you only get a 1.6x speedup, it means you optimized the wrong part of the program! Suppose 20% of the program is responsible for 80% of the execution time. If you speed up this bit four times, you end up with a 2.5x speedup (and not a 1.18x speedup as the article tries to imply).
- taneq 9y ago
  Basically this. The article makes it sound like "optimizing half the program" means "optimizing half of the codebase", not "optimizing a part of the codebase that consumes 50% of the execution time." This quote is especially misleading: > Amdahl’s Law is telling us that to significantly improve the speed of a system, we’ll have to improve a very large fraction of it. No, it's telling us we'll have to improve the part of it which is responsible for a very large fraction of the execution time. Often this is only a tiny amount of the overall codebase.
pif 9y ago
Software is not Modern or Ancient. Software is either Good (i.e. it works as it is supposed to do AND it is easily maintainable and modifiable) or Bad. NASA clearly needs Good software, and indeed the article never mentions arbitrary deadlines.

247 points by astdb 9y ago

bluedino 9y ago

I'd like to see how this compares to, SpaceX for example

dsfyu404ed 9y ago
When it comes to physical components SpaceX has a habit of selecting an off the shelf component and asking the vendor if they can get a modified version that includes some changes that make it capable of living in their application. That's great for reducing cost (or increasing quality or buying time, they're all related) but in the long term it often leads to spaghetti. Considering the time pressure they're perpetually under and how Elon is known to run his organizations I would suspect their software products are no different.
- brianwawok 9y ago
  On that note, what language(s) are SpaceX, Tesla built on? Job postings I see a mix of C# and golang. I cannot believe either a Model X or a rocket is rockin C#...
  - 0xffff2 9y ago
    Probably not on the rocket itself, but C# is a very common language for ancillary testing tools in the industrial automation world. I wouldn't be surprised if a lot of their test infrastructure is written in C#.
    - w_t_payne 9y ago
      Pity. I've been using Python a lot for test infrastructure and it fits really well.
  - elteto 9y ago
    The software team did an AMA on reddit a while back: https://www.reddit.com/r/IAmA/comments/1853ap/we_are_spacex_software_engineers_we_launch/ https://www.reddit.com/r/IAmA/comments/1853ap/we_are_spacex_... Basically it's C/C++ (as expected) for critical stuff, like the rocket or dragon, and C# for all enterprise systems. Plus an assorted set of other languages/technologies depending on team and project.
mcguire 9y ago
The last I heard, from outside the loop, there was considerable concern about how they were going to get man rated.

lisper 9y ago

This article is very much at odds with my 15-year (1988-2000, 2001-2004) experience writing software at NASA (albeit at JPL, which is to say the unmanned space program, but still NASA). In my experience there was just as much politics, marketing hype, and general bullshit as I've seen in the commercial world. It was actually pretty amazing to me some times that things worked at all, let alone that they worked as well as they did. I think the real secret at NASA is not that they are better at writing code, but that they write it slowly (development cycles are measured in years) and they test the living shit out of it. Then, once it's working, they don't change it except under extreme pressure. But while it's being developed it's just as much of a sausage factory as anything else.

sethrin 9y ago
Does that mean that as long as you test the shit out of it, it doesn't matter how it gets done?
- lifeisstillgood 9y ago
  That seems a reasonable position - yes.
  - pbhjpbhj 9y ago
    Reasonable if you ignore maintainability and standards compliance. Not a coder, but if you used an idiom that worked because of a compiler quirk then that could lead to problems down the line but related to maintenance per se, hence why I added the second part.
    - SomeHacker44 9y ago
      In any mission critical situation the entire tool chain is just as locked down and controlled as the "source code," even down to the silicon.
      - olewhalehunter 9y ago

alexanderdmitri 9y ago

Here's a paper[1] for a space conference in Montreal written a few years ago on DO-178 standards and applying the FAA's certification process to space systems development (DO-178 sets the bar for "testing the shit" out of software, I've seen projects held up for years because it's like approaching a limit f(n) -> L (where f(n) is the software development cycle and L is getting a non-bought DER to sign off on it) proving your system to its requirements). [1] http://articles.adsabs.harvard.edu/cgi-bin/nph-iarticle_query?2013ESASP.715E..63D&data_type=PDF_HIGH&whole_paper=YES&type=PRINTER&filetype=.pdf http://articles.adsabs.harvard.edu/cgi-bin/nph-iarticle_quer...

nickpsecurity 9y ago
DO-178B was one of best things to happen in software assurance. It was a non-prescriptive method of certification that forced precise requirements, design, and code with reviews of each and tracability between them. The high cost of certification and certification failure led to a booming ecosystem of supporting tools (esp static analysis or runtimes) and pre-made components (esp RTOS's & middleware). Some companies even specialized in handling the administrative overhead. The result was that certified systems usually had very, high quality. Meanwhile, there were people arguing on HN and other places about whether a certification could ever be useful for software. Anyone reading the above paper should skip to the summary to get a high-level view of the overhead and resulting benefits. The author of the paper agrees with me that the certification is beneficial. The gist is you keep things simple, think about everything in lifecycle, test everything in lifecycle, throw every analysis tool you can at the design or code, esp take your time to develop the thing, and document all of this in a way where any aspect is easy to verify by a 3rd party. That's how high-assurance systems are done. They usually work, too. Some work for a long, long, long time. :)
- mcguire 9y ago
  DO-178C has a formal methods follow-on. I haven't seen it, though.
- ptero 9y ago
  Applied very narrowly to the right problem, maybe. But I have also seen DO-178B to lead to a completely non-maintainable mess. That mess helps trace each statement to a requirement, but leads to code that is unreadable to anyone used to normal code. It is not that "no extra code" and exactly meeting requirements is a bad model in itself. However writing code in a way that makes it convenient to prove the above encourages unnatural coding styles and programmatic contortions. YMMV however.

deleted 9y ago

[deleted]

Judgmentality 9y ago

I always found this article(1) about writing code for NASA fascinating. I'm sure a lot has changed in the 20 years since it was written, but it's such a stark contrast to what I'm used to. (1) https://www.fastcompany.com/28121/they-write-right-stuff https://www.fastcompany.com/28121/they-write-right-stuff

mcguire 9y ago
20 years ago: * 420,000 LOC * 260 people * $35,000,000 / year
- Judgmentality 9y ago
  There's something to be said about doing more with less.

rav 9y ago

The article severely misrepresents Amdahl's law, in particular in the sentence: > Though you took half of your entire program, and optimized it to be four times faster, the overall system is only 1.6x faster. This application of Amdahl's law doesn't tell you anything about what happens when you take half of the program; it tells you what happens when you take half of the execution time. If you only get a 1.6x speedup, it means you optimized the wrong part of the program! Suppose 20% of the program is responsible for 80% of the execution time. If you speed up this bit four times, you end up with a 2.5x speedup (and not a 1.18x speedup as the article tries to imply).

taneq 9y ago
Basically this. The article makes it sound like "optimizing half the program" means "optimizing half of the codebase", not "optimizing a part of the codebase that consumes 50% of the execution time." This quote is especially misleading: > Amdahl’s Law is telling us that to significantly improve the speed of a system, we’ll have to improve a very large fraction of it. No, it's telling us we'll have to improve the part of it which is responsible for a very large fraction of the execution time. Often this is only a tiny amount of the overall codebase.

pif 9y ago

Software is not Modern or Ancient. Software is either Good (i.e. it works as it is supposed to do AND it is easily maintainable and modifiable) or Bad. NASA clearly needs Good software, and indeed the article never mentions arbitrary deadlines.