4 ms·

That's a dangerous line and I don't think it's correct. Software I write shouldn't be relied on in critical situations. If someone makes that decision then it's

That's a dangerous line and I don't think it's correct. Software I write shouldn't be relied on in critical situations. If someone makes that decision then it's on them not on me. The line should be where a person tells others that they can rely on the software with their lives - as in the integrator for the end product. Even if I was working on the software for self driving, the same thing would apply - if I wrote some alpha level stuff for the internal demonstration and some manager decided "good enough, ship it", they should be liable for that decision. (Because I wouldn't be able to stop them / may have already left by then)

presentation 2y ago
To be fair maybe the software you write shouldn’t be relied on in critical situations but in this case the only place this software could be used in are critical situations
- viraptor 2y ago
  Ultimately - yes. But as I mentioned, the fact it's sold as ready for critical situations doesn't mean the developers thought/said it's ready.
  - elric 2y ago
    I think it should be fairly obvious that it's not the individual developers who are responsible/liable. In critical systems there is a whole chain of liability. That one guy in Nebraska who thanklessly maintains some open source lib that BigCorp is using in their car should obviously not be liable.
    - f1shy 2y ago
      It depends. If you do bad sw and skip reviews and processes, you may be liable. Even if you are told to do something, if you know is wrong, you should say it. Right now I’m in middle of s*t because of I spoked up.
      - Filligree 2y ago
        > Right now I’m in middle of s*t because of I spoked up. And you believe that, despite experiencing what happens if you speak up? We shouldn’t simultaneously require people to take heroic responsibility, while also leaving them high and dry if they do.
        f1shy 2y ago
        I do believe I am responsible. I recognize I’am now in a position that I can speak without fear. If I get fired I would make a party tbh.
  - gmueckl 2y ago
    But someone slapped that label on it and made a pinky promise that it's true. That person needs to accept liability if things go wrong. If person A is loud and clear that something isn't ready, but person B tells the customer otherwise, B is at fault. Look, there are well established procedures in a lot of industries where products are relied on to keep people safe. They all require quite rigorous development and certification processes and sneaking untested alpha quality software through such a process would be actively malicious and quite possibly criminal in and of itself, at least in some industries.
    - viraptor 2y ago
      This is the beginning of the thread https://news.ycombinator.com/item?id=41891164 https://news.ycombinator.com/item?id=41891164 You're in violent agreement with me ;)
      - latexr 2y ago
        No, the beginning of the thread is earlier. And with that context it seems clear to me that the “you” in the post you linked means “the company”, not “the individual software developer”. No one else in your replies seems confused by that, we all understand self-driving software wasn’t written by a single person that has ultimate decision power within a company.
        viraptor 2y ago
        If the message said "you release software", or "approve" or "produce", or something like that, sure. But it said "you write software" - and I don't think that can apply to a company, because writing is what individuals do. But yeah, maybe that's not what the author meant.
        latexr 2y ago
        > and I don't think that can apply to a company, because writing is what individuals do. By that token, no action could ever apply to a company—including approving, producing, or releasing—since it is a legal entity, a concept, not a physical thing. For all those actions there was a person actually doing it in the name of the company. It’s perfectly normal to say, for example, “GenericCorp wrote a press-release about their new product”.
kergonath 2y ago
It’s not that complicated or outlandish. That’s how most engineering fields work. If a building collapses because of design flaws, then the builders and architects can be held responsible. Hell, if a car crashes because of a design or assembly flaw, the manufacturer is held responsible. Why should self-driving software be any different? If the software is not reliable enough, then don’t use it in a context where it could kill people.
- krisoft 2y ago
  I think the example here is that the designer draws a bridge for a railway model, and someone decides to use the same design and sends real locomotives across it. Is the original designer (who neither intended nor could have foreseen this) liable in your understanding?
  - ndsipa_pomu 2y ago
    That's a ridiculous argument. If a construction firm takes an arbitrary design and then tries to build it in a totally different environment and for a different purpose, then the construction firm is liable, not the original designer. It'd be like Boeing taking a child's paper aeroplane design and making a passenger jet out of it and then blaming the child when it inevitably fails.
    - deleted 2y ago
      [deleted]
    - wongarsu 2y ago
      Or alternatively, if Boeing uses wood screws to attach an airplane door and the screw fails that's on Boeing, not the airline, pilot or screw manufacturer. But if it's sold as aerospace-grade attachment bolt with attachments for safety wire and a spec sheet that suggests the required loads are within design parameters then it's the bolt manufacturers fault when it fails, and they might have to answer for any deaths resulting from that. Unless Boeing knew or should have known that the bolts weren't actually as good as claimed, then the buck passes back to them Of course that's wildly oversimplifying and multiple entities can be at fault at once. My point is that these are normal things considered in regular engineering and manufacturing
sigh_again 2y ago
>Software I write shouldn't be relied on in critical situations. Then don't write software to be used in things that are literally always critical situations, like cars.

presentation 2y ago

To be fair maybe the software you write shouldn’t be relied on in critical situations but in this case the only place this software could be used in are critical situations

viraptor 2y ago
Ultimately - yes. But as I mentioned, the fact it's sold as ready for critical situations doesn't mean the developers thought/said it's ready.
- elric 2y ago
  I think it should be fairly obvious that it's not the individual developers who are responsible/liable. In critical systems there is a whole chain of liability. That one guy in Nebraska who thanklessly maintains some open source lib that BigCorp is using in their car should obviously not be liable.
  - f1shy 2y ago
    It depends. If you do bad sw and skip reviews and processes, you may be liable. Even if you are told to do something, if you know is wrong, you should say it. Right now I’m in middle of s*t because of I spoked up.
    - Filligree 2y ago
      > Right now I’m in middle of s*t because of I spoked up. And you believe that, despite experiencing what happens if you speak up? We shouldn’t simultaneously require people to take heroic responsibility, while also leaving them high and dry if they do.
      - f1shy 2y ago
        I do believe I am responsible. I recognize I’am now in a position that I can speak without fear. If I get fired I would make a party tbh.
- gmueckl 2y ago
  But someone slapped that label on it and made a pinky promise that it's true. That person needs to accept liability if things go wrong. If person A is loud and clear that something isn't ready, but person B tells the customer otherwise, B is at fault. Look, there are well established procedures in a lot of industries where products are relied on to keep people safe. They all require quite rigorous development and certification processes and sneaking untested alpha quality software through such a process would be actively malicious and quite possibly criminal in and of itself, at least in some industries.
  - viraptor 2y ago
    This is the beginning of the thread https://news.ycombinator.com/item?id=41891164 https://news.ycombinator.com/item?id=41891164 You're in violent agreement with me ;)
    - latexr 2y ago
      No, the beginning of the thread is earlier. And with that context it seems clear to me that the “you” in the post you linked means “the company”, not “the individual software developer”. No one else in your replies seems confused by that, we all understand self-driving software wasn’t written by a single person that has ultimate decision power within a company.
      - viraptor 2y ago
        If the message said "you release software", or "approve" or "produce", or something like that, sure. But it said "you write software" - and I don't think that can apply to a company, because writing is what individuals do. But yeah, maybe that's not what the author meant.
        latexr 2y ago
        > and I don't think that can apply to a company, because writing is what individuals do. By that token, no action could ever apply to a company—including approving, producing, or releasing—since it is a legal entity, a concept, not a physical thing. For all those actions there was a person actually doing it in the name of the company. It’s perfectly normal to say, for example, “GenericCorp wrote a press-release about their new product”.

kergonath 2y ago

It’s not that complicated or outlandish. That’s how most engineering fields work. If a building collapses because of design flaws, then the builders and architects can be held responsible. Hell, if a car crashes because of a design or assembly flaw, the manufacturer is held responsible. Why should self-driving software be any different? If the software is not reliable enough, then don’t use it in a context where it could kill people.

krisoft 2y ago
I think the example here is that the designer draws a bridge for a railway model, and someone decides to use the same design and sends real locomotives across it. Is the original designer (who neither intended nor could have foreseen this) liable in your understanding?
- ndsipa_pomu 2y ago
  That's a ridiculous argument. If a construction firm takes an arbitrary design and then tries to build it in a totally different environment and for a different purpose, then the construction firm is liable, not the original designer. It'd be like Boeing taking a child's paper aeroplane design and making a passenger jet out of it and then blaming the child when it inevitably fails.
  - deleted 2y ago
    [deleted]
  - wongarsu 2y ago
    Or alternatively, if Boeing uses wood screws to attach an airplane door and the screw fails that's on Boeing, not the airline, pilot or screw manufacturer. But if it's sold as aerospace-grade attachment bolt with attachments for safety wire and a spec sheet that suggests the required loads are within design parameters then it's the bolt manufacturers fault when it fails, and they might have to answer for any deaths resulting from that. Unless Boeing knew or should have known that the bolts weren't actually as good as claimed, then the buck passes back to them Of course that's wildly oversimplifying and multiple entities can be at fault at once. My point is that these are normal things considered in regular engineering and manufacturing

sigh_again 2y ago

>Software I write shouldn't be relied on in critical situations. Then don't write software to be used in things that are literally always critical situations, like cars.