3 ms·

Of course it matters. That’s the “end effect” portion of any proper risk management process. The severity and likelihood of various end effects drives improveme

Of course it matters. That’s the “end effect” portion of any proper risk management process. The severity and likelihood of various end effects drives improvement in the design, creation, and usage of the item.

mewpmewp2 3y ago
But why are people surprised then that in most software the process is not as thorough as in aerospace or elsewhere with physical things. Most of the time physical devices failing will have much more severe consequences, than a website going down for an hour. In some cases there are APIs which many other critical systems use and frequently those APIs do have much more rigorous process in making sure that they are reliable. All of that seems regulated by governments and markets to me. They have promises of reliability and big financial penalties if they were to fail on that reliability.
- godelski 3y ago
  > But why are people surprised then that in most software the process is not as thorough as in aerospace or elsewhere with physical things. Because the traditional engineers do this in a lot of stuff and not just high risk domains. You can probably find somewhere an FEA analysis on your TV's legs and where the high stress points are and where it'll break. > They have promises of reliability and big financial penalties if they were to fail on that reliability. Ehhhhh really? Or if everyone does a shit job do we just take it? Most people aren't tech literate. Hell, most apps aren't built by tech literate people. I have a laundry app I have to use and the rooms aren't in any logical order, just randomly placed. Are you tech literate if you don't know a sort function exists? Or if you don't know people like to read things in a sorted order? If you can't google the answer? There's lots of systems that have really shit designs in them and as discussed elsewhere in the thread I don't think companies are having big financial consequences for their lightbulbs not working when the wifi goes down because the people purchasing them aren't aware that such a insane thing would happen. If you're not very tech literate you wouldn't even think that's a possible outcome. So don't just rely on markets because econ 101 isn't enough.
- syndicatedjelly 3y ago
  > But why are people surprised then that in most software the process is not as thorough as in aerospace or elsewhere with physical things. Outside of engineering, few people have ever even heard the phrase “risk management”, let alone understand the theory behind it. It’s really hard to argue that a developer can consider themself an engineer, when they don’t bother understanding what happens when the thing they “engineered” fails. Yes this applies to web applications too. If someone can’t access an application that gives them directions, it’s possible that they will be stranded. If a healthcare website fails to load a list of available physicians, it’s possible the user gives up and doesn’t seek medical care. The current prevailing attitude in web dev of “who cares” is really a testament to how little most web developers and real engineers have in common. Software engineers working on safety-critical systems deserve that title because they actually have to understand software risks, design appropriate mitigations, and prove their effectiveness.
  - godelski 3y ago
    Thanks for this comment. Clearly in my original comment I made I failed to articulate how useful failure analysis is for even highly mundane things. So if we take this as a post mortum then I think we can see the failure mode of my language. The big question though is if this is within reasonable bounds of noise as over emphasizing mundane examples may result in the concept not making an adequate impression on the importance of the topic. I did have a large number of points with no responses prior to receiving misinterpretations but that's a noisy metric itself so now I'm asking a third party consultant if they have any thoughts on how I can improve my communication. And not to say there aren't plenty of software engineers that do failure analysis. I mean we do have exceptions, exit codes, traces, and even QA (though many hate them). But I'm not under the impression that the idea is as ingrained in the average software engineer as it is in the average engineer. Having taught CS I've been surprised at how much less time is spent on this topic and how less explicit it is usually discussed when compared to other engineering disciples. The same goes for ethics which I think go hand in hand here. My impression is that this is because when software harms it is often less tangible and less direct than in other engineering disciplines where it's obvious how a bridge failing causes substantial harm and economic impact but where a failure to protect against injection in your not mainstream platform also causes large damage because people reuse passwords and that passing the buck to user error is unacceptable given that it's expected behavior from the user (work on expected behavior and account for unexpected behavior but never rely on domain knowledge unless under special circumstances). We all know everyone reuses their Gmail and bank password on websites like Netflix or fucking neopets. We wouldn't have as frequent front page posts about users getting accounts unlocked only due to the fact that they were able to go viral if software engineers better planned for failures in the system. Do Gmail devs really deeply think about what happens when someone loses access to their account? If there's a means of recovery if they're devices are also taken or destroyed? That this solution works in practice, not just theory? Do they audit and dogfood this process to ensure it's working properly and understand how it fails and the consequences of that? I'm assuming no because we see people locked out of these systems ("these" because I'm generalizing beyond Gmail)

mewpmewp2 3y ago

But why are people surprised then that in most software the process is not as thorough as in aerospace or elsewhere with physical things. Most of the time physical devices failing will have much more severe consequences, than a website going down for an hour. In some cases there are APIs which many other critical systems use and frequently those APIs do have much more rigorous process in making sure that they are reliable. All of that seems regulated by governments and markets to me. They have promises of reliability and big financial penalties if they were to fail on that reliability.

godelski 3y ago
> But why are people surprised then that in most software the process is not as thorough as in aerospace or elsewhere with physical things. Because the traditional engineers do this in a lot of stuff and not just high risk domains. You can probably find somewhere an FEA analysis on your TV's legs and where the high stress points are and where it'll break. > They have promises of reliability and big financial penalties if they were to fail on that reliability. Ehhhhh really? Or if everyone does a shit job do we just take it? Most people aren't tech literate. Hell, most apps aren't built by tech literate people. I have a laundry app I have to use and the rooms aren't in any logical order, just randomly placed. Are you tech literate if you don't know a sort function exists? Or if you don't know people like to read things in a sorted order? If you can't google the answer? There's lots of systems that have really shit designs in them and as discussed elsewhere in the thread I don't think companies are having big financial consequences for their lightbulbs not working when the wifi goes down because the people purchasing them aren't aware that such a insane thing would happen. If you're not very tech literate you wouldn't even think that's a possible outcome. So don't just rely on markets because econ 101 isn't enough.
syndicatedjelly 3y ago
> But why are people surprised then that in most software the process is not as thorough as in aerospace or elsewhere with physical things. Outside of engineering, few people have ever even heard the phrase “risk management”, let alone understand the theory behind it. It’s really hard to argue that a developer can consider themself an engineer, when they don’t bother understanding what happens when the thing they “engineered” fails. Yes this applies to web applications too. If someone can’t access an application that gives them directions, it’s possible that they will be stranded. If a healthcare website fails to load a list of available physicians, it’s possible the user gives up and doesn’t seek medical care. The current prevailing attitude in web dev of “who cares” is really a testament to how little most web developers and real engineers have in common. Software engineers working on safety-critical systems deserve that title because they actually have to understand software risks, design appropriate mitigations, and prove their effectiveness.
- godelski 3y ago
  Thanks for this comment. Clearly in my original comment I made I failed to articulate how useful failure analysis is for even highly mundane things. So if we take this as a post mortum then I think we can see the failure mode of my language. The big question though is if this is within reasonable bounds of noise as over emphasizing mundane examples may result in the concept not making an adequate impression on the importance of the topic. I did have a large number of points with no responses prior to receiving misinterpretations but that's a noisy metric itself so now I'm asking a third party consultant if they have any thoughts on how I can improve my communication. And not to say there aren't plenty of software engineers that do failure analysis. I mean we do have exceptions, exit codes, traces, and even QA (though many hate them). But I'm not under the impression that the idea is as ingrained in the average software engineer as it is in the average engineer. Having taught CS I've been surprised at how much less time is spent on this topic and how less explicit it is usually discussed when compared to other engineering disciples. The same goes for ethics which I think go hand in hand here. My impression is that this is because when software harms it is often less tangible and less direct than in other engineering disciplines where it's obvious how a bridge failing causes substantial harm and economic impact but where a failure to protect against injection in your not mainstream platform also causes large damage because people reuse passwords and that passing the buck to user error is unacceptable given that it's expected behavior from the user (work on expected behavior and account for unexpected behavior but never rely on domain knowledge unless under special circumstances). We all know everyone reuses their Gmail and bank password on websites like Netflix or fucking neopets. We wouldn't have as frequent front page posts about users getting accounts unlocked only due to the fact that they were able to go viral if software engineers better planned for failures in the system. Do Gmail devs really deeply think about what happens when someone loses access to their account? If there's a means of recovery if they're devices are also taken or destroyed? That this solution works in practice, not just theory? Do they audit and dogfood this process to ensure it's working properly and understand how it fails and the consequences of that? I'm assuming no because we see people locked out of these systems ("these" because I'm generalizing beyond Gmail)