3 ms·

A real scenario that actually happened to someone I was in jail with (and they were jailed for this). They were (legally drunk) passenger in a car on a remote s

A real scenario that actually happened to someone I was in jail with (and they were jailed for this). They were (legally drunk) passenger in a car on a remote section of highway late at night. (Sober, diabetic) Driver's blood sugar too low and driver fell unconscious. Passenger grabbed wheel, pulled car to shoulder. Dialed 911. Ambulance to take too long. Moves driver over and gets behind wheel. Gets most of way to hospital. Pulled over by cops. Cops arrest him and drive unconscious person to hospital. I introduced him to the defense of "necessity" while in jail: https://en.wikipedia.org/wiki/Necessity_(criminal_law) https://en.wikipedia.org/wiki/Necessity_(criminal_law) My point is.. the driver might well have died if the car had totally refused to function because he was over the limit.

kelnos 5y ago
And that's the thing: these sorts of software systems don't and cannot ever understand edge cases. Short of full AI that can reason about the situation and make a decision (even then, it may make the wrong decision, just as people make wrong decisions sometimes), this is always going to be a problem. An emergency override button can mitigate this problem, but then you have to weigh how often that override will be abused, and what to do about that.