6 ms·
Yes, this situation is in the realm of the plausible, which has little to do with my comment. I'm not arguing against an analysis of the decisions made by a se
by saidajigumi 10y ago
Yes, this situation is in the realm of the plausible, which has little to do with my comment. I'm not arguing against an analysis of the decisions made by a self-driving system. That's essential. I'm arguing against applying ethical models that utilize knowledge outside the realm of the knowledge or "reasoning" capacity of the drive algorithms.
To continue your example, I'll pose that any system with unilateral behavior of self-endangerment is extremely problematic. Motivating example: assume that self-driving cars will always avoid head-on collisions with other vehicles, even if there's no safe alternative. Including driving off a cliff. Attack: manual driver drives down the wrong side of the road (or maybe the middle of a two-lane road), forcing all of the self-driving cars off the cliff during evening commute hours to a hillside residential neighborhood. I suspect all systems that don't hew to some baseline of self-preservation are subject to similar modes of attack.
- schoen 10y agoThere's a whole kind of unpleasant but interesting topic about attacks against behavior of self-driving cars. I think I saw an earlier HN thread that suggested that they might put occupants at greater risk of kidnapping because the cars could refuse to take evasive actions that endanger or injure a kidnapper (or just violate a traffic law). This risk might be farfetched, but it may illustrate the concern about people knowing the limits that self-driving cars will respect and then exploiting those limits in some way.
- schoen 10y ago> I think I saw an earlier HN thread that suggested I might also have heard about some of these problems from Brad Templeton, who describes "Abusing robocar collision-avoidance" among many other interesting problems at http://www.templetons.com/brad/robocars/downsides.html http://www.templetons.com/brad/robocars/downsides.html I don't remember if he also mentioned the kidnapping scenario when I heard him present about this.
- imtringued 10y ago>I think I saw an earlier HN thread that suggested that they might put occupants at greater risk of kidnapping because the cars could refuse to take evasive actions that endanger or injure a kidnapper What alternative would a human take? Would they kill the kidnappers by driving over them?
- schoen 10y agoI guess that's the simplest one that a self-driving car's software might not approve of. Or intentionally collide with a kidnapper's vehicle, or just make a prohibited turn or run a red light to try to escape. These scenarios are kind of vague because they don't make it clear whether the self-driving car lets a human driver take over and take full responsibility.
- kbart 10y ago"Would they kill the kidnappers by driving over them?" Of course, in many countries that would count as self defense (if kidnappers were armed and pointed guns towards your car).
- DavidWanjiru 10y agoI think a human would drive over a kidnapper. Say your car is bullet proof. A man in a ski mask wielding a machine gun jumps in front of the road and orders you to stop. You refuse, he starts shooting at your car as you get near him (this is why I made your car bullet proof, to keep you alive but unequivocally demonstrate clear and present danger to you). In such a scenario, if your only escape route is over the dead body of the man in a ski mask, I bet a human will take that option.