5 ms·

I don't doubt for a minute that NASA has highly competent programmers who write bulletproof ultra-reliable software, but they didn't get their knowledge or skil

I don't doubt for a minute that NASA has highly competent programmers who write bulletproof ultra-reliable software, but they didn't get their knowledge or skill from this tedious bureaucratic tome. After skimming the document for 15 minutes, I didn't find a single thing that would be insightful to experienced software developers or their managers. Things like, "Increased complexity means increased errors at all levels of development." It should be possible to write a useful book about software safety, but this is not it.

pekk 10y ago
A remarkable number of people with impressive resumes don't just lack these insights, they will actively argue with them.
- fit2rule 10y ago
  Yeah, it seems sort of discomfiting to hear a claim that 'experienced professionals will sniff at this and not learn a thing from it'. You're not supposed to learn anything from this - its there so that the standard is set, and professionals - experienced, or otherwise - can refer to the standard, and measure performance against it. An 'experienced professional' doesn't treat every document like a Bible from which new enlightenment can be gained. Often you need such things as this manual in order to quiet the rabble and give the mob something else to argue about, instead of eating themselves - as often happens when standards are not set and expectations not set and met.
nabla9 10y ago
I think many people fail to understand that safety critical software can't rely on "highly competent programmers". No matter how competent, programmers can't be trusted to write safety critical code. Safety critical software is all about the process and organization. The process must be so good that you can hire total newbie to write code and none of his sloppy work would get into the final product. What you need is software engineer who writes 20 lines of code and can then justify each of the 100 lines in a resulting spreadsheet that is full MC/DC code coverage in processor instruction level. Then you need two testing engineers who can do the same for code that others have written. Then you need everyone to go trough every change, justify them, rewrite tests and provide reports.
- Spooky23 10y ago
  Article speaking at a 30,000 ft level about what you describe. http://www.fastcompany.com/28121/they-write-right-stuff http://www.fastcompany.com/28121/they-write-right-stuff
- ben_jones 10y ago
  To extrapolate on this the skillset to identify "highly competent programmers" AND populate your organization with them exclusively either does not exist or at least is unreliable enough that it can never be assumed as a permanent state of your organization: as proven by all long-term software projects and the organizations that created them.
  - termain 10y ago
    Including those at NASA.
- somethingsimple 10y ago
  > What you need is software engineer who writes 20 lines of code and can then justify each of the 100 lines in a resulting spreadsheet that is full MC/DC code coverage in processor instruction level. Then you need two testing engineers who can do the same for code that others have written. Then you need everyone to go trough every change, justify them, rewrite tests and provide reports. Working like this would be my dream job. Typical software "engineering" has close to no process, no rigor, it's just smart kids pumping code like mad. And then people are upset when major issues arise.
  - DarkTree 10y ago
    Yeah I think it mainly comes down to who the code is serving. When people's lives are on the line, it is extremely valuable to make bullet-proof code. However, when you have to get a product out to users to start fueling growth for VCs.. well then let's get that code out the door ASAP. Extending the process in the latter case in exchange for more robust code doesn't always make sense.
- 10y ago
hga 10y ago
See this current HN item on writing Space Shuttle software: https://news.ycombinator.com/item?id=12014248 https://news.ycombinator.com/item?id=12014248 For an early thorough review of that team, there was a great CACM article I read in 1982-3 as I recall, it was written not long after the first launch in 1981, mentions how they changed/touched 70%? of the software after that launch, and in simulation found a possible error that would do something asymmetric with the SRBs, which would have been fatal to the shuttle (although maybe not the pilots, who in the first 4 flights had SR-71 derived ejection systems).
fredgrott 10y ago
You have to remember how this process was developed.. At the time of the Apollo program MIT was contracted to write the navigation software..if it was done their research way we would have never landed on the moon as they wanted an all approach. Instead what was negotiated between the engineers and astronauts was an approach where NASA radio signals carried the navigation stuff and the nav computers in the command module and lunar modules were backups and were not fully a full navigation of the full mission.
lisper 10y ago
> After skimming the document for 15 minutes, I didn't find a single thing that would be insightful to experienced software developers or their managers That's because the purpose of this document is not to advance the process of producing safe software, it is to facilitate the process of assigning blame when something goes wrong, and in particular to help deflect blame from upper management and onto a scapegoat lower down on the food chain. This document is not operative in the forward direction, i.e. it is not the case that people read it and acquire knowledge which they then use to write better software. It is operative in the reverse direction, i.e. when something goes wrong, upper management scours the document to find a place where whatever went wrong deviated from the rules. They then say, "Look, it's not out fault, we had the right rules in place. It is the fault of whoever deviated from the rules." This is not a phenomenon unique to NASA, BTW. Most large bureaucratic organizations eventually develop this dynamic.
- cpeterso 10y ago
  I read a somewhat tongue-in-cheek blog post about "blame-oriented software design". I can't find it now, but it had actually useful advice on design techniques like logging and API precondition checks to avoid blame for other people's bugs. :)

pekk 10y ago

A remarkable number of people with impressive resumes don't just lack these insights, they will actively argue with them.

fit2rule 10y ago
Yeah, it seems sort of discomfiting to hear a claim that 'experienced professionals will sniff at this and not learn a thing from it'. You're not supposed to learn anything from this - its there so that the standard is set, and professionals - experienced, or otherwise - can refer to the standard, and measure performance against it. An 'experienced professional' doesn't treat every document like a Bible from which new enlightenment can be gained. Often you need such things as this manual in order to quiet the rabble and give the mob something else to argue about, instead of eating themselves - as often happens when standards are not set and expectations not set and met.

nabla9 10y ago

I think many people fail to understand that safety critical software can't rely on "highly competent programmers". No matter how competent, programmers can't be trusted to write safety critical code. Safety critical software is all about the process and organization. The process must be so good that you can hire total newbie to write code and none of his sloppy work would get into the final product. What you need is software engineer who writes 20 lines of code and can then justify each of the 100 lines in a resulting spreadsheet that is full MC/DC code coverage in processor instruction level. Then you need two testing engineers who can do the same for code that others have written. Then you need everyone to go trough every change, justify them, rewrite tests and provide reports.

Spooky23 10y ago
Article speaking at a 30,000 ft level about what you describe. http://www.fastcompany.com/28121/they-write-right-stuff http://www.fastcompany.com/28121/they-write-right-stuff
ben_jones 10y ago
To extrapolate on this the skillset to identify "highly competent programmers" AND populate your organization with them exclusively either does not exist or at least is unreliable enough that it can never be assumed as a permanent state of your organization: as proven by all long-term software projects and the organizations that created them.
- termain 10y ago
  Including those at NASA.
somethingsimple 10y ago
> What you need is software engineer who writes 20 lines of code and can then justify each of the 100 lines in a resulting spreadsheet that is full MC/DC code coverage in processor instruction level. Then you need two testing engineers who can do the same for code that others have written. Then you need everyone to go trough every change, justify them, rewrite tests and provide reports. Working like this would be my dream job. Typical software "engineering" has close to no process, no rigor, it's just smart kids pumping code like mad. And then people are upset when major issues arise.
- DarkTree 10y ago
  Yeah I think it mainly comes down to who the code is serving. When people's lives are on the line, it is extremely valuable to make bullet-proof code. However, when you have to get a product out to users to start fueling growth for VCs.. well then let's get that code out the door ASAP. Extending the process in the latter case in exchange for more robust code doesn't always make sense.
10y ago

hga 10y ago

See this current HN item on writing Space Shuttle software: https://news.ycombinator.com/item?id=12014248 https://news.ycombinator.com/item?id=12014248 For an early thorough review of that team, there was a great CACM article I read in 1982-3 as I recall, it was written not long after the first launch in 1981, mentions how they changed/touched 70%? of the software after that launch, and in simulation found a possible error that would do something asymmetric with the SRBs, which would have been fatal to the shuttle (although maybe not the pilots, who in the first 4 flights had SR-71 derived ejection systems).

fredgrott 10y ago

You have to remember how this process was developed.. At the time of the Apollo program MIT was contracted to write the navigation software..if it was done their research way we would have never landed on the moon as they wanted an all approach. Instead what was negotiated between the engineers and astronauts was an approach where NASA radio signals carried the navigation stuff and the nav computers in the command module and lunar modules were backups and were not fully a full navigation of the full mission.

lisper 10y ago

> After skimming the document for 15 minutes, I didn't find a single thing that would be insightful to experienced software developers or their managers That's because the purpose of this document is not to advance the process of producing safe software, it is to facilitate the process of assigning blame when something goes wrong, and in particular to help deflect blame from upper management and onto a scapegoat lower down on the food chain. This document is not operative in the forward direction, i.e. it is not the case that people read it and acquire knowledge which they then use to write better software. It is operative in the reverse direction, i.e. when something goes wrong, upper management scours the document to find a place where whatever went wrong deviated from the rules. They then say, "Look, it's not out fault, we had the right rules in place. It is the fault of whoever deviated from the rules." This is not a phenomenon unique to NASA, BTW. Most large bureaucratic organizations eventually develop this dynamic.

cpeterso 10y ago
I read a somewhat tongue-in-cheek blog post about "blame-oriented software design". I can't find it now, but it had actually useful advice on design techniques like logging and API precondition checks to avoid blame for other people's bugs. :)