13 ms·
The Power of Ten – Rules for Developing Safety Critical Code
- ranko 10y agoThe original paper describing and justifying these rules in more detail is at http://spinroot.com/gerard/pdf/P10.pdf http://spinroot.com/gerard/pdf/P10.pdf, and the official document is at http://lars-lab.jpl.nasa.gov/JPL_Coding_Standard_C.pdf http://lars-lab.jpl.nasa.gov/JPL_Coding_Standard_C.pdf
- sctb 10y agoThanks! We updated the link from http://www.rankred.com/nasa-coding-rules http://www.rankred.com/nasa-coding-rules.
- guntars 10y agoI understand the need for linking to the original source, but I just looked at both and the latter is arguably harder to read.
- teh_klev 10y agoYes, but the rankred article hasn't reproduced the full paper, it's just cherry picked and summarised in a way that you would have no idea if you missed something worthwhile. The original paper isn't that terrible to read. Also the amount of third party cookie crap that site tries to deposit in my browser is crazy. That's why HN prefers source material, not blogspam stuff like this which adds nothing new and often removes possibly useful information.
- thomaslee 10y agoAny Debian users interested in Spin ought to be able to `apt-get install spin` sometime soon: https://tracker.debian.org/pkg/spin https://tracker.debian.org/pkg/spin It's already landed in unstable, testing shouldn't be too far away.
- hackits 10y agoFound the rule `There shall be no use of dynamic memory allocation after task initialization.` They must use pre-allocated string buffers for some of the task's. Though I figured if you're doing a flight controller for a jet fighter or sate-light you don't need a lot of string parsing.
- kwhitefoot 10y agoAlmost all of the programs that I have ever read parse strings far too often. Almost all scientific and engineering software can eliminate it except at the UI and database.
- TorKlingberg 10y agoI have written software for mobile network base stations where there were hardly any strings. I think it was only process names and log messages, both of which were string constants. I imagine unmanned space software is similar; there is little need for a text UI on the spacecraft itself.
- TickleSteve 10y agodynamic memory allocation is very much frowned upon in embedded systems, not just for strings. Everything should be static and deterministic at all times. This is the easiest/only way to ensure you have no resource issues. You should always (statically) allocate for maximum/worst case.. because you have analysed your worst-case, haven't you?
- petra 10y agoHow do you analyze worst case ? don't you need to know what calls what, up to what depth, and that's dynamic by nature ?
- TickleSteve 10y agoIt all comes from the defined requirements and specifications. i.e. "You shall handle x messages in y milliseconds." from that, you derive your worst case buffer size given that you can service that buffer every 'z' milliseconds at most (Note, that involves a hard-real-time requirement as it is a bounded maximum time).
- keithnz 10y agoThese are basically a subset of rules that MISRA defines for C programming.
- elcapitan 10y ago> Do not use [..] direct or indirect recursion. Ok.. I get it that they don't want their C programmers to do that, but do they also mean that this is to "complex" for normal developers to implement in a fault-free manner?
- corecoder 10y agoThey mean that it's hard to write a static analysis tool that can determine if functions using recursion will ever terminate or not.
- elcapitan 10y agoJust out of curiosity, why is that harder than writing a static analysis that determines if a loop with condition will finish?
- adrianratnapala 10y agoRule #2 is "Fixed Upper Bound for Loops". Although they define this flexibily as "It should be possible for a verification tool to prove statically that a preset upper-bound on the number of iteration of a loop can’t be exceeded." So any loops where they can in fact write a verifier is simple enough.
- Jtsummers 10y agoThe sort of software these rules are intended for run in constrained environments. The other question besides "Will this loop/recursion terminate?" is "Will we blow up the stack?". Ignoring the time required to establish a new stackframe (versus a mere jump for a loop), the creation of a stack frame uses more memory. The less memory you can use, while still producing maintainable code, the better. And an easy way to assist with this is to remove recursion. You could make the case for tail recursion, which a sufficiently advanced compiler will/can/should turn into a mere jump, like a loop, because it is. But this isn't guaranteed in the C standard and so should not be relied upon. And then there's Rule #2. Loops are bounded, the index variable is supposed to only be altered by the loop structure (so: `for(i = 0; i < MAX; i++)`, that increment is the only place `i` should be altered). By providing a compile-time max, and incrementing (deterministically) the index variable, you ensure that static analysis of the loop terminating (or not) is possible.
- atemerev 10y agoAll of these make perfect sense, but I would have allowed tail recursion (and only that). Its properties are well-defined and boundedness can be easily proven (also it is easy to distinguish tail and non-tail recursive calls).
- gpvos 10y agoThey're using C. Many C compilers don't optimize tail recursion away, and even for those that do, it is not generally obvious from the C syntax whether something is a tail call or not.
- Eerie 10y agoThis is being re-posted approximately every two weeks. Why?
- thearn4 10y agoSpeaking from experience, the biggest problem with NASA's software engineering requirements (http://nodis3.gsfc.nasa.gov/displayDir.cfm?t=NPR&c=7150&s=2B http://nodis3.gsfc.nasa.gov/displayDir.cfm?t=NPR&c=7150&s=2B) is the way that they tend to feed down into non-safety critical projects. It's getting better though.
- hackits 10y agoHaving a look at the engineering requirements there are guidelines that I've seen that are used by medium to large scale organisation just simply to manage the project and code base. Some of them seem to be universal such as statistic code analysis, heave use of sticking to a sub-set of the language that is well defined (Deterministic behavior). Sticking to a over-all coding convention and naming convention. Modulation of the code-base. Avoidance to thread's mutating the state of shared objects between threads (Share state between threads only in a single location eg.. IPC). Similar thing happened with java API with the designers attempting to make the language library thread safe. They soon discovered that was a real bad idea and backed off from that very fast. At work I joke with my collegue's about the java framework they use they've now moved back to static enum's and static control id's for web pages instead of dynamically generating them on the fly. It's to a point where I feel like I'm programming win32 gui's all over again.
- pjmlp 10y agoSadly quality is highly disregarded in our field. I dream of the day when not making use of contracts, static analysis and type based programming is seen as quality smell and not something that only a few are allowed to make use of.
- lambdadmitry 10y agoIt's not "disregarded", it's eclipsed by a need to ship often and ship a lot. There is a tendency to overstate amount of problems that come from bugs because they are painful to debug. Therefore, somewhat experienced programmers are often too defensive and suspect to paralysis by analysis (of which uber-complicated and rigid typing is a sort). Even if some advanced typing system will save a few days of debugging after refactoring down the line, it doesn't matter if a competitor ships buggy RoR-based systems a few months earlier. Advanced typing/static analysis is not a pinnacle of software engineering, it's a set of training wheels. Useful? Yeah. But sometimes you may need to skip them.
- petra 10y agoIf we're talking about high reliability code, one thing claimed about Haskell, is "if it compiles, it has no bugs". How close is it to the truth ? And how close are we to having that kind of capability for real-time programming(even assuming we're willing to forsake protability, community, and maximum efficiency to some extent ) ?
- msoucy 10y agoHaskell's type system protects from a certain class of bug, but it can't prevent logic errors - something like using < instead of > won't trigger a build error, but it certainly is a bug.
- lmm 10y agoSometimes you can express the relevant relationship in the type system. Just adding types to an existing program won't tell you much, but if you work with the type system it's possible to move a lot of your business logic there.
- _ph_ 10y agoThe stricter and more advanced a type system is, the more this comes close to the truth. Of course, no compiler can check that the function you implemented is the function you wanted to implement, but it can quite check that it can reliably compute the function you typed in. The question hinges, on how much errors in your program express themselves as type errors too.
- pjmlp 10y agoI imagine you will find this paper interesting "Haskell vs. Ada vs. C++ vs. Awk vs. ... An Experiment in Software Prototyping Productivity" http://haskell.cs.yale.edu/?post_type=publication&p=366 http://haskell.cs.yale.edu/?post_type=publication&p=366
- AnimalMuppet 10y agoI would imagine, though, that "prototyping" and "safety critical" are on opposite ends of the spectrum...
- garethrees 10y agoIt looks to me as though the recommendation that RankRed has titled "Rule No. 5 — Low Assertion Density" would be better described as "High Assertion Density" — the recommendation is for a minimum of two assertions per function (and functions are supposed to be short per rule 4). The recommendations look good to me and (with one caveat) correspond to rules that I apply when writing C code with a high reliability requirement. My one caveat is in "Rule No. 8 – Limited Use of Preprocessor" which bans all complex uses of the preprocessor. The problem is that it is common in C to encounter situations where the only way to avoid lots of code duplication is to store a table of facts in a macro definition and use the preprocessor to expand those facts into the relevant bits of code in each place where they are used. So in these situations you face a trade-off between the risks due to complex preprocessor use, and the risks due to code duplication (namely, a maintainer might change a fact in one place where it is used but fail to change it in another). My experience is that the risks due to code duplication are very high, and so it's worth the risk of using complex preprocessor macros to avoid them. The risks can be mitigated by implementing the necessary macros in a structured way to keep the complexity under control: http://garethrees.org/2007/04/24/relational-macros/ http://garethrees.org/2007/04/24/relational-macros/
- titzer 10y agoAssertions in code cut both ways. Sometimes they can be great; telling you exactly which assumed invariant is violated. However, that doesn't tell you where or how the invariant was violated. Sometimes assertions are just crutches for lazy programming. Instead of handling of a very valid (corner) case, some people just assert that it doesn't happen. Lo and behold, years later, it does happen. And those years later, the context is completely lost. How easy is it to handle the corner case now? Hard. In this situation, instead of simply asserting that it doesn't happen, it might be warranted to just assume that it _can_ happen and handle the case. Then, followed up with coverage testing, one needs to (actually try hard) come up with an input that triggers it. Assertions can also have bugs. The worst is trying to figure out which assertions are really valuable and which aren't. Shotgun-spraying assertions in the code is not a good strategy in general.
- 10y ago
- ballooney 10y agoThis is a very interesting talk about writing the software for the Curiosity Mars mission by the head of Software Reliability at JPL, Gerard Holzmann, who also wrote these 10 rules: https://vimeo.com/84991949 https://vimeo.com/84991949
- Bromskloss 10y agoNice tool setup. I would like to work under those circumstances.
- vog 10y agoThis is an all-time classic. Too bad that the article doesn't mention the original paper: http://pixelscommander.com/wp-content/uploads/2014/12/P10.pdf http://pixelscommander.com/wp-content/uploads/2014/12/P10.pd... Some interesting HN discussions around applying these NASA coding standards to JavaScript: https://news.ycombinator.com/item?id=8856226 https://news.ycombinator.com/item?id=8856226
- atsaloli 10y agoI posted a write-up of Dr. Holzmann's talk "Mars Code" which he gave at USENIX Hot Topics in System Dependability '12 (http://www.verticalsysadmin.com/making_robust_software/ http://www.verticalsysadmin.com/making_robust_software/), about how NASA/JPL writes and tests software that survives the rigors of interplanetary travel; planetary entry, descent, and landing; and operation in another world. :)
- lordnacho 10y agoHow does that fixed upper bound on loops work? If there's an array of dynamic size that needs to be looped over, how do you do that?
- gernerz 10y agoI cant speak for NASA, but in general if you are writing extremely safety critical code you simply don't use dynamically sized things. In addition to safety, the dynamically sized objects cause issues for the real-time constraints on these system as well.
- robmccoll 10y ago#define MAX_NUM_OBJS 100 for(int i = 0; i < num_objs && i < MAX_NUM_OBJS; i++) Combined with the no dynamic allocation rule, you can guarantee that the number of elements in the array must be less than some maximum since you have limited dedicated space for storing them.
- lordnacho 10y agoThere might not be a valid object at every location in the preallocated memory. Say you have a list of visible satellites, and the number of them can change as they go in/out of view. There's never more than 100, but sometimes there's 50 and sometimes 60. I supposed you could have an object with an INVALID flag, which the loop can use in its logic?
- robmccoll 10y agoThat's a good point. If you do have a situation where the number of objects is somewhat dynamic, you could use layers of protection such as (re)initializing the unused objects to a known safe state, adding a flag as you mentioned, adding sentinel values to the end, and at the very least you do have the guarantee that you aren't running off into memory that was actually dedicated to another purpose and contains fundamentally different data. If you are doing something really dynamic, you could embed two linked lists (or an embedded list and a free index stack or even two packed stacks). The reason for stacks over lists is that lists can accidentally become cycles.
- optforfon 10y ago- What tools do they use for error checking/linting? - Well... why C? If correctness is so important... surely there are better languages than C - not only better type systems, but also theorem proving and other fancy modern features. It doesn't even have to be garbage collected. - at the very least are they using language extensions? even gcc primitves would help!
- delinka 10y agoThis formatting doesn't match the rest of HN which makes it difficult to read. Is this a bug somewhere?
- deleted 10y ago[deleted]
- drauh 10y agoThey seem to be different Unicode codepoints: FULLWIDTH LATIN SMALL LETTER
- Bromskloss 10y agoI wonder, is that really something that should go into a character set?
- astrange 10y agoIt was needed for JIS compatibility, since they encode halfwidth and fullwidth versions of kana and English characters, because having different fonts for them was too expensive. Also it's needed for this tweet. https://twitter.com/pisscop/status/67709754859995136 https://twitter.com/pisscop/status/67709754859995136
- jamie_ca 10y agoSome kind of unicode copy/paste hijinx, probably coming from a PDF somewhere.
- 10y ago
- Symmetry 10y agoRegarding Rule 4 - No Large Function - I think John Carmack had a pretty good explanation on why that's not really a good thing: http://number-none.com/blow/john_carmack_on_inlined_code.html http://number-none.com/blow/john_carmack_on_inlined_code.htm...
- GregBuchholz 10y ago>The fly-by-wire flight software for the Saab Gripen (a lightweight >fighter) went a step further. It disallowed both subroutine calls and >backward branches, except for the one at the bottom of the main loop. >Control flow went forward only. Sometimes one piece of code had to leave >a note for a later piece telling it what to do, but this worked out well >for testing: all data was allocated statically, and monitoring those >variables gave a clear picture of most everything the software was doing. >The software did only the bare essentials, and of course, they were >serious about thorough ground testing. >No bug has ever been found in the "released for flight" versions of that code. I was pretty excited to learn more about that style, and then I came across... https://www.flightglobal.com/FlightPDFArchive/1989/1989%20-%200734.PDF https://www.flightglobal.com/FlightPDFArchive/1989/1989%20-%... ...but I'd still like to know more about this style of programming, and languages that facilitate programming in that manner.
- dragonwriter 10y agoIf I read that article right, it sounds like the software fault attributed to the crash is that the "control laws" that the software implemented were improperly chosen, not that the software did not function as intended. Which is to say, a serious problem, but if we take "bug" to mean a difference between designed function and realized function, not a "bug", and therefore not something which should tarnish the record of the coding style to which the bug-free nature of the Grippen software is attributed. The problem seems to have been at a different level than coding.
- safeignorance 10y agoYou're correct to point out that the coding style isn't to blame for the software fault. And IMO The last paragraph of the article hints at the most probable fundamental cause. But I just don't buy that this was not a software fault. It clearly was a case of a faulty software specification. > The problem seems to have been at a different level than coding. This makes me queasy. Software engineers working on these sorts of systems -- -- or at least a few senior ones -- should understand enough of the domain to say "this spec is not adequate" or "bad things might happen under conditions xyz; what's the correct behavior in these cases?". And of course all software should notice and then degrade gracefully when assumptions are observably violated. To absolve the software engineer of any responsibility for understanding the context surrounding his software is to wrongly assume there's not much to software engineering beyond programming to someone else's spec.
- geophile 10y agoIn rule 5 (about assertions), why the comparison to true? Why is "if (<boolean expression> == true)" preferable to "if (<boolean expresson>)"?
- Udo 10y agoI assume it's a habit to make sure you communicate to the type checker: "I expect this expression to be boolean".
- DominikR 10y agoRecently there was a thread here where many argued against some company owner that has set the objective that the code created by his employees should be as simple as possible. Often the arguments boiled down to: It looks only clever if the skill level of your employees is too low, you should work on that. When I look now at the rules that NASA enforces, it seems to me that their objective is to prevent bugs by not allowing code that is overly complex and I doubt it's because their engineers are not clever enough. In fact I believe that it takes a lot of cleverness to create code that seems so simple that you don't even need to read the documentation.
- smaddox 10y agoIt seems that people often confuse complex with clever or impressive. Impressive is unifying the falling of an apple with the orbiting of the planets into three simple universal laws. Complexity is an emergent phenomenon. It occurs when there are too many interactions to track. To make something simple is to take all of that complexity and to understand and model it as the interaction of a few simple rules repeated over and over and over.
- DominikR 10y agoGreat analogy, I agree!
- finishingmove 10y ago> a function shouldn’t have more than 60 lines of code. I started having headache spikes after reading this sentence that refers to keeping functions "short", in the article about safety critical programs.
- paulryanrogers 10y agoWhile the embedded world does have some hard depth limits, my experience has been that the readability of breaking up a function are worth it. This seems especially true when the nested functions are carefully named.
- crpatino 10y agoI agree that making functions shorter does not automatically make them safer, but I do see how having a well designed "main" function in the ballpark of ~80LOC which calls another 10 auxiliary (and inlined) minor functions of ~15-25LOC would be easier to reason about than an equivalent function of ~220LOC. The fact that you could safisfy the business requirement with a single function of ~140LOC does not mean the other approach is more verbose, but that you are only thinking on the happy path, and a bunch of edge cases are left undefined.
- DasIch 10y agoKeep in mind that we're talking about C code here. Error handling etc. blows that up quite significantly.
- lisper 10y agoRule 0: don't write safety-critical code in C.
- AnimalMuppet 10y agoNice snark, I guess, but... did you actually read the article? One of the reasons for C is that the tooling around it gives you much more support. For safety critical code, you need that, because programmers make mistakes, no matter what the language. So if you write safety critical code in Haskell, say, you won't have a large number of the holes that C gives you. But for safety critical systems, their rules say that you can't write code that has even the possibility of falling into those holes. That makes C much safer. That leaves you with a safer C, plus tooling, vs. Haskell (or whatever your choice is) without the tooling. So, snark aside, what is your actual alternative that you recommend?
- lisper 10y ago> did you actually read the article? Yes. I used to work with the person who wrote it. > One of the reasons for C is that the tooling around it gives you much more support. That's because C is an unsafe language, so to write safe code you need that tooling. If you start with a safe language the need for that tooling simply evaporates because many of the mistakes that the tooling is designed to catch are simply not possible. > For safety critical code, you need that, because programmers make mistakes, no matter what the language. That's certainly true, but many of those mistakes are mistakes in the rendering of the desired semantics into code. Neither rules nor tooling can catch those kinds of mistakes. > But for safety critical systems, their rules say that you can't write code that has even the possibility of falling into those holes. That makes C much safer. No, it doesn't. All of the rules that are not specific to C can be equally well applied to any other language. > So, snark aside, what is your actual alternative that you recommend? Good heavens, just about anything is better than C if what you care about is safety. Ada. Java. Scheme. Haskell. OCaml. Python. Go. Swift. Common Lisp.
- AnimalMuppet 10y ago
- avindroth 10y agoMy first two years at undergrad would have been so much easier if I had read this piece of document. I remember using a quadruple pointer (char ) in one of my systems classes. Also, after having dabbled in Haskell, I know the value of static analyzers. That is like good practices on steroids. And you start to avoid these errors, and start developing better practices. Change the environment to change your habits.
- fazkan 10y agoI think the compiler is as much responsible for secure code as the code itself, tools should be updated regularly with these core concepts in mind...These 10 points, I figure are just the starting points, there are a lot of other places where the flow of program changed by abusing simple keywords such as volatile. Which is where the correct use of compiler, serve as a nightwatch to stop any undefined behavior from crossing the wall....
- rttyyuiidii 10y agoDo you want Watch Free live adult webcam models through skype. cam guys top cam babes are live at adult webcam chat shows,live cam sex porn, free live nude shows visit at: http://bit.ly/29KxXAQ http://bit.ly/29KxXAQ
- rttyyuiidii 10y agoDo you want Watch Free live adult webcam models through skype. cam guys top cam babes are live at adult webcam chat shows,live cam sex porn, free live nude shows visit at: http://bit.ly/29KxXAQ http://bit.ly/29KxXAQ