3 ms·

You, an engineer at a major aircraft manufacturer that isn't Boeing, have been working after hours with some of your colleagues on a hobby project to add some m

You, an engineer at a major aircraft manufacturer that isn't Boeing, have been working after hours with some of your colleagues on a hobby project to add some modern safety features to an older model of small private plane, because you regard it as unsafe even though it still has a government certification and you got into this field because you want to save lives. Your "prototype" is a plane from the original manufacturer with no physical modifications but a software patch to use data from sensors the plane already had to prevent the computer from getting confused under high wind conditions in a way that has already caused two fatal crashes. Now you have to fly somewhere and your options for a plane are the one with the history of fatal crashes or the same one with your modifications, and it's windy today. Which plane are you getting on?

sarusso 10mo ago
This example is so right. Including the parallel with what happened with those two aircrafts.
amrocha 10mo ago
Definitely not the untested code I wrote myself! Are you kidding me? How many times have you unwillingly introduced bugs into a code base you didn’t fully understand? That’s basically table stakes for software engineering.
- AnthonyMouse 10mo ago
  > Definitely not the untested code I wrote myself! Nobody said it was untested. > How many times have you unwillingly introduced bugs into a code base you didn’t fully understand? That’s basically table stakes for software engineering. Which applies just the same to the people the company hired to do it, and now we're back to "the people with a stronger incentive to get it right are the people who die if it goes wrong".
  - tstrimple 10mo ago
    Tested how? With 100% "unit test" coverage? I can certainly see how a random person on the internet might be highly motivated and actually talented enough to contribute to these sorts of projects. But they don't have the budget and resources that commercial entities have. They don't have the same due diligence requirements. They don't have the same liability. If I use a commercial device unaltered, it's the company's fault if the device fucks up or is defective and causes harm. If I install random internet software on my medical device and it fucks up and causes harm, it's my fault. I say this as someone who might modify my own medical devices because I'm so fucking jaded over the capitalist march towards enshitification and maximizing profit over human lives. There is simply no way random folks on the internet can test these types of systems to any reliable degree. It requires rigorous testing across hundreds to thousands of test cases. They at best can give you the recipe that works well for them and the few people that have voluntarily tried their version. That doesn't scale and certainly isn't any safer than corporate solutions.
    - AnthonyMouse 10mo ago
      > But they don't have the budget and resources that commercial entities have. Everyone is standing on the shoulders of giants. You're not going from stone tools to jet engines in a month, but you could fix a bug in one in that time. > They don't have the same due diligence requirements. They don't have the same liability. Things that exist to try to mitigate the misalignment of incentives that comes from paying someone else to create something you depend on. Better for the incentives to align to begin with. Notice also that these things are floors, not ceilings. The company is only required to do the minimum. You can exceed it by as much as you like. > If I use a commercial device unaltered, it's the company's fault if the device fucks up or is defective and causes harm. If I install random internet software on my medical device and it fucks up and causes harm, it's my fault. And then if the community version fixes a bug that would have killed you and you stick with the commercial version you can sue them for killing you. Except that you're dead. > There is simply no way random folks on the internet can test these types of systems to any reliable degree. Basically the entire population is on the internet, so the set of them includes all the people doing it for a corporation. Are they going to forget how to do their jobs when they go home, or when they or a member of their family gets issued another company's device and they want it to be right?
  - amrocha 10mo ago
    I can’t tell if you seriously think a random person writing code in their basement is equivalent to a company that has access to API docs, design specs, actual test hardware, the expertise of a ton of engineers that have worked on the project and understand how it can go wrong, not to mention all the regulations and verifications they’re subject to. But if you do then wow. That really puts in perspective the kind of people that use hacker news. I’m gonna be more selective about who I bother replying to going forward.
    - AnthonyMouse 10mo ago
      > I can’t tell if you seriously think a random person writing code in their basement is equivalent to a company that has access to API docs, design specs Are you saying not having those things is dangerous? They should be required to publish all of that for safety-critical devices then. > actual test hardware Why would arbitrary people be unable to buy test hardware? Again something to be addressed if true rather than used as an excuse. > the expertise of a ton of engineers that have worked on the project and understand how it can go wrong Do they not have internet access? If they don't even work for the company anymore then that could be the only way to access that information. Literally something which is happening on the linked Reddit page. > not to mention all the regulations and verifications they’re subject to. Regulations are for preventing someone else from harming you. You don't need a government incentive to protect you from yourself, you already come with that incentive.

sarusso 10mo ago

This example is so right. Including the parallel with what happened with those two aircrafts.

amrocha 10mo ago

Definitely not the untested code I wrote myself! Are you kidding me? How many times have you unwillingly introduced bugs into a code base you didn’t fully understand? That’s basically table stakes for software engineering.

AnthonyMouse 10mo ago
> Definitely not the untested code I wrote myself! Nobody said it was untested. > How many times have you unwillingly introduced bugs into a code base you didn’t fully understand? That’s basically table stakes for software engineering. Which applies just the same to the people the company hired to do it, and now we're back to "the people with a stronger incentive to get it right are the people who die if it goes wrong".
- tstrimple 10mo ago
  Tested how? With 100% "unit test" coverage? I can certainly see how a random person on the internet might be highly motivated and actually talented enough to contribute to these sorts of projects. But they don't have the budget and resources that commercial entities have. They don't have the same due diligence requirements. They don't have the same liability. If I use a commercial device unaltered, it's the company's fault if the device fucks up or is defective and causes harm. If I install random internet software on my medical device and it fucks up and causes harm, it's my fault. I say this as someone who might modify my own medical devices because I'm so fucking jaded over the capitalist march towards enshitification and maximizing profit over human lives. There is simply no way random folks on the internet can test these types of systems to any reliable degree. It requires rigorous testing across hundreds to thousands of test cases. They at best can give you the recipe that works well for them and the few people that have voluntarily tried their version. That doesn't scale and certainly isn't any safer than corporate solutions.
  - AnthonyMouse 10mo ago
    > But they don't have the budget and resources that commercial entities have. Everyone is standing on the shoulders of giants. You're not going from stone tools to jet engines in a month, but you could fix a bug in one in that time. > They don't have the same due diligence requirements. They don't have the same liability. Things that exist to try to mitigate the misalignment of incentives that comes from paying someone else to create something you depend on. Better for the incentives to align to begin with. Notice also that these things are floors, not ceilings. The company is only required to do the minimum. You can exceed it by as much as you like. > If I use a commercial device unaltered, it's the company's fault if the device fucks up or is defective and causes harm. If I install random internet software on my medical device and it fucks up and causes harm, it's my fault. And then if the community version fixes a bug that would have killed you and you stick with the commercial version you can sue them for killing you. Except that you're dead. > There is simply no way random folks on the internet can test these types of systems to any reliable degree. Basically the entire population is on the internet, so the set of them includes all the people doing it for a corporation. Are they going to forget how to do their jobs when they go home, or when they or a member of their family gets issued another company's device and they want it to be right?
- amrocha 10mo ago
  I can’t tell if you seriously think a random person writing code in their basement is equivalent to a company that has access to API docs, design specs, actual test hardware, the expertise of a ton of engineers that have worked on the project and understand how it can go wrong, not to mention all the regulations and verifications they’re subject to. But if you do then wow. That really puts in perspective the kind of people that use hacker news. I’m gonna be more selective about who I bother replying to going forward.
  - AnthonyMouse 10mo ago
    > I can’t tell if you seriously think a random person writing code in their basement is equivalent to a company that has access to API docs, design specs Are you saying not having those things is dangerous? They should be required to publish all of that for safety-critical devices then. > actual test hardware Why would arbitrary people be unable to buy test hardware? Again something to be addressed if true rather than used as an excuse. > the expertise of a ton of engineers that have worked on the project and understand how it can go wrong Do they not have internet access? If they don't even work for the company anymore then that could be the only way to access that information. Literally something which is happening on the linked Reddit page. > not to mention all the regulations and verifications they’re subject to. Regulations are for preventing someone else from harming you. You don't need a government incentive to protect you from yourself, you already come with that incentive.