3 ms·

> Any time you bring up the idea of accountability to “engineers” for defects in their software that cause serious, real life harm, you get a litany of excuses

by Kamq 3y ago

> Any time you bring up the idea of accountability to “engineers” for defects in their software that cause serious, real life harm, you get a litany of excuses in response I'm generally against it, because I don't trust anyone to regulate this in a sane manner, but I'm open to discussion. If we're going have engineers sign off on things and be legally liable, there has to be a context that you're signing off on them to be used in as well. If a physical engineer signs off on a pedestrian bridge, and then people decide to drive semi-trucks over it, there's generally no liability for the original engineer. Same situation if an engineer signs off on a sturdy bridge, and someone makes a bunch of changes to materials without telling them. These seem roughly analogous to someone opening up something designed for private use to the open internet, and someone making changes to a function without telling the original engineer, respectively. These have to be excluded, or nobody can ever sign off on anything.

ryandrake 3y ago
Yep, context and environment is important. You're signing off on your product being used in a particular context and in a particular environment. The semi-truck driving over a pedestrian bridge is a great example. Unfortunately for much of software engineering, our "environment" is the open Internet where there are largely invisible, international, adversarial attackers working 24 hours a day, seven days a week. With Internet-connected software we can't just say "Oh, this software's intended environment is a clean-room LAN with no connected devices! That's all I'm signing off." That's not reality. As for your example, companies should really, really have a hard conversation about taking a software designed for privacy use and just opening it up to the Internet without hardening it sufficiently. Accountability would help make that conversation possible.
- JumpCrisscross 3y ago
  > semi-truck driving over a pedestrian bridge is a great example It also incentivises the engineer to clearly document their design’s limits. Imagine if software sales had that much transparency.

by Kamq 3y ago

ryandrake 3y ago

Yep, context and environment is important. You're signing off on your product being used in a particular context and in a particular environment. The semi-truck driving over a pedestrian bridge is a great example. Unfortunately for much of software engineering, our "environment" is the open Internet where there are largely invisible, international, adversarial attackers working 24 hours a day, seven days a week. With Internet-connected software we can't just say "Oh, this software's intended environment is a clean-room LAN with no connected devices! That's all I'm signing off." That's not reality. As for your example, companies should really, really have a hard conversation about taking a software designed for privacy use and just opening it up to the Internet without hardening it sufficiently. Accountability would help make that conversation possible.

JumpCrisscross 3y ago
> semi-truck driving over a pedestrian bridge is a great example It also incentivises the engineer to clearly document their design’s limits. Imagine if software sales had that much transparency.