5 ms·

A lot of people look at safety critical development standards to try and copy process bits for quality. In reality, 90% of the quality benefits come from sittin

A lot of people look at safety critical development standards to try and copy process bits for quality. In reality, 90% of the quality benefits come from sitting down to think about the software and its role in the overall system. You don't need all the fancy methodologies and expensive tools. It's also the main benefit you get from formal methods. I've found that a quality process that starts with "you need to comprehensively understand what you're engineering" is almost universally a non-starter for anyone not already using these things. Putting together an exhaustive list of all the ways code interacts with the outside world is hard. If a few engineers actually manage it, they're rarely empowered to make meaningful decisions on whether the consequences of failures are acceptable or fix things if they're not.

tehjoker 10mo ago
I think the main benefit of these standards is that when someone proposes a project, the level gets evaluated and either enough (and appropriate) resources are allocated or it is killed in an ideal world.
- AlotOfReading 10mo ago
  You'd hope. That's not always my experience. What I often see is cutting random bits off the development plan until the resource constraints are nominally satisfied, without much regard for whether the resulting plan is sensible. That's if there's a plan. Sometimes these systems get randomly assigned a level based on vibes, with the expectation that someone will later go back and fix the level if it's incorrect. This works about as well as commented TODOs.
MobiusHorizons 10mo ago
I also generally find that people looking for “best practices” to follow are trying to avoid that “sitting down to think about the software and its role in the overall system” piece.
kqr 10mo ago
It doesn't help that many of the popular methodologies focus entirely on failures. They ask a bunch of questions in the style of "how likely is it that this part fails?" "what happens if it fails?" "how can we reduce the risk of it failing?" etc. But software never fails[1] so that's the wrong approach to start from! Much better to do as you say and think about the software and its role in the system. There are more and less formal ways to do this, but it's definitely better than taking a component view.
- ryandrake 10mo ago
  FYI you added a [1] but didn't add the link to whatever you were going to reference!
  - teddyh 10mo ago
    It could have been this: “The reason is that, in other fields [than software], people have to deal with the perversity of matter. [When] you are designing circuits or cars or chemicals, you have to face the fact that these physical substances will do what they do, not what they are supposed to do. We in software don't have that problem, and that makes it tremendously easier. We are designing a collection of idealized mathematical parts which have definitions. They do exactly what they are defined to do. And so there are many problems we [programmers] don't have. For instance, if we put an ‘if’ statement inside of a ‘while’ statement, we don't have to worry about whether the ‘if’ statement can get enough power to run at the speed it's going to run. We don't have to worry about whether it will run at a speed that generates radio frequency interference and induces wrong values in some other parts of the data. We don't have to worry about whether it will loop at a speed that causes a resonance and eventually the ‘if’ statement will vibrate against the ‘while’ statement and one of them will crack. We don't have to worry that chemicals in the environment will get into the boundary between the if statement and the while statement and corrode them, and cause a bad connection. We don't have to worry that other chemicals will get on them and cause a short-circuit. We don't have to worry about whether the heat can be dissipated from this ‘if’ statement through the surrounding ‘while’ statement. We don't have to worry about whether the ‘while’ statement would cause so much voltage drop that the ‘if’ statement won't function correctly. When you look at the value of a variable you don't have to worry about whether you've referenced that variable so many times that you exceed the fan-out limit. You don't have to worry about how much capacitance there is in a certain variable and how much time it will take to store the value in it. All these things are defined a way, the system is defined to function in a certain way, and it always does. The physical computer might malfunction, but that's not the program's fault. So, because of all these problems we don't have to deal with, our field is tremendously easier.” — Richard Stallman, 2001: <https://www.gnu.org/philosophy/stallman-mec-india.html#conf9 https://www.gnu.org/philosophy/stallman-mec-india.html#conf9>
mubbicles 10mo ago
Another good document for military standards for software safety is AOP-52. Has some fun anecdotes in it. My favorite being the nuclear certified supersonic aircraft having a latent defect discovered during integration of a new subsystem. Turns out all of the onboard flight computers crashed at the transition from sub to supersonic, thankfully the aircraft had enough inertia to "ride through" all of their flight computers simultaneously crashing during the transonic boundary. Moral of that story is your software people need to have the vocabulary to understand the physical properties of the system they're working on.
jcims 10mo ago
>Putting together an exhaustive list of all the ways code interacts with the outside world is hard. Maintaining it over time is even harder.
exe34 10mo ago
it's cargo culting. we see the same thing with "agile", which is often used as an excuse to just do what they were going to do anyway. they want the benefits, and are willing to do everything except the things that actually help.
inamberclad 10mo ago
Absolutely. If you look at an extensively used standard like DO-178C for avionics, it really says very little about how to program. Instead, the emphasis is on making sure that the software has implemented system level requirements correctly.
trklausss 10mo ago
I see the fancy methodologies and processes as the way of streamlining what you have to do in order to "sit down to think about the software", particularly in teams of more than one developer. Most of it happens, as always, at the interface. So these methodologies help you manage these interfaces between people, machine and product.

tehjoker 10mo ago

I think the main benefit of these standards is that when someone proposes a project, the level gets evaluated and either enough (and appropriate) resources are allocated or it is killed in an ideal world.

AlotOfReading 10mo ago
You'd hope. That's not always my experience. What I often see is cutting random bits off the development plan until the resource constraints are nominally satisfied, without much regard for whether the resulting plan is sensible. That's if there's a plan. Sometimes these systems get randomly assigned a level based on vibes, with the expectation that someone will later go back and fix the level if it's incorrect. This works about as well as commented TODOs.

MobiusHorizons 10mo ago

I also generally find that people looking for “best practices” to follow are trying to avoid that “sitting down to think about the software and its role in the overall system” piece.

kqr 10mo ago

It doesn't help that many of the popular methodologies focus entirely on failures. They ask a bunch of questions in the style of "how likely is it that this part fails?" "what happens if it fails?" "how can we reduce the risk of it failing?" etc. But software never fails[1] so that's the wrong approach to start from! Much better to do as you say and think about the software and its role in the system. There are more and less formal ways to do this, but it's definitely better than taking a component view.

ryandrake 10mo ago
FYI you added a [1] but didn't add the link to whatever you were going to reference!
- teddyh 10mo ago
  It could have been this: “The reason is that, in other fields [than software], people have to deal with the perversity of matter. [When] you are designing circuits or cars or chemicals, you have to face the fact that these physical substances will do what they do, not what they are supposed to do. We in software don't have that problem, and that makes it tremendously easier. We are designing a collection of idealized mathematical parts which have definitions. They do exactly what they are defined to do. And so there are many problems we [programmers] don't have. For instance, if we put an ‘if’ statement inside of a ‘while’ statement, we don't have to worry about whether the ‘if’ statement can get enough power to run at the speed it's going to run. We don't have to worry about whether it will run at a speed that generates radio frequency interference and induces wrong values in some other parts of the data. We don't have to worry about whether it will loop at a speed that causes a resonance and eventually the ‘if’ statement will vibrate against the ‘while’ statement and one of them will crack. We don't have to worry that chemicals in the environment will get into the boundary between the if statement and the while statement and corrode them, and cause a bad connection. We don't have to worry that other chemicals will get on them and cause a short-circuit. We don't have to worry about whether the heat can be dissipated from this ‘if’ statement through the surrounding ‘while’ statement. We don't have to worry about whether the ‘while’ statement would cause so much voltage drop that the ‘if’ statement won't function correctly. When you look at the value of a variable you don't have to worry about whether you've referenced that variable so many times that you exceed the fan-out limit. You don't have to worry about how much capacitance there is in a certain variable and how much time it will take to store the value in it. All these things are defined a way, the system is defined to function in a certain way, and it always does. The physical computer might malfunction, but that's not the program's fault. So, because of all these problems we don't have to deal with, our field is tremendously easier.” — Richard Stallman, 2001: <https://www.gnu.org/philosophy/stallman-mec-india.html#conf9 https://www.gnu.org/philosophy/stallman-mec-india.html#conf9>

mubbicles 10mo ago

Another good document for military standards for software safety is AOP-52. Has some fun anecdotes in it. My favorite being the nuclear certified supersonic aircraft having a latent defect discovered during integration of a new subsystem. Turns out all of the onboard flight computers crashed at the transition from sub to supersonic, thankfully the aircraft had enough inertia to "ride through" all of their flight computers simultaneously crashing during the transonic boundary. Moral of that story is your software people need to have the vocabulary to understand the physical properties of the system they're working on.

jcims 10mo ago

>Putting together an exhaustive list of all the ways code interacts with the outside world is hard. Maintaining it over time is even harder.

exe34 10mo ago

it's cargo culting. we see the same thing with "agile", which is often used as an excuse to just do what they were going to do anyway. they want the benefits, and are willing to do everything except the things that actually help.

inamberclad 10mo ago

Absolutely. If you look at an extensively used standard like DO-178C for avionics, it really says very little about how to program. Instead, the emphasis is on making sure that the software has implemented system level requirements correctly.

trklausss 10mo ago

I see the fancy methodologies and processes as the way of streamlining what you have to do in order to "sit down to think about the software", particularly in teams of more than one developer. Most of it happens, as always, at the interface. So these methodologies help you manage these interfaces between people, machine and product.