5 ms·

Should autonomous cars kill their owner to save multiple other lives?

24 points by maxcan 12y ago

serf 12y ago
now there's a selling point. "Hyper-aggressive BMW AI takes out 15 pedestrians and causes $250,000 in collateral city-wide damage. Reason cited : danger to owner's career prospects from possible late business lunch arrival. Thankfully the owner is reported to be safe (and punctual)."
smrtinsert 12y ago
Yes, and knowing that, I'm sure everyone will buy an autonomous car, so that it will save the many, and not their family.
- EpicEng 12y ago
  I don't know why this is getting downvoted. Sorry, but our most basic human instinct is that of survival. By that I mean my and my family's survival, not yours. I'm not saying that I wouldn't sacrifice myself to save many other people (no one really knows what they would do until they actually have to make that call), but I'm not going to buy something that will sacrifice my wife and child because a computer thinks it's the best outcome.
  - fivethree 12y ago
    Congrats, you've exposed the failings of human logic and ethics!
dllthomas 12y ago
It seems to me that - socially - we should regard this as the responsibility of the driver rather than the automaker. They chose to drive in that place at that time, they maintain their vehicle, and they have chosen some balance between safety and heroism (either by tweaking a setting or by picking what car they buy). It's the driver's agency at work.
- krapp 12y ago
  At first, sure. But the goal of autonomous cars is to eventually remove driver agency altogether. What happens when all cars are fully autonomous? Will they collectively weight their passengers by "killability", perhaps based on age, insurance premium, marital status, social media status, etc?
  - dllthomas 12y ago
    I don't follow. You're proposing there will be driverless cars pursuing their own agenda, as opposed to the agenda of some human or group of humans who have - in some sense - deployed them? That sounds unlikely before the robot uprising.
    - krapp 12y ago
      I don't know about "pursuing their own agenda" but making their own decisions - yes, probably. There isn't much point to autonomous cars which are essentially drones.
      - dllthomas 12y ago
        Decisions don't create agency. My router makes decisions about what packets to route (and where) and what packets to drop. It does so based on a policy I set and those decisions are an extension of my agency. If I configure my car to prefer saving me to saving 100 schoolchildren (or pick a car with that configuration over the alternative), I'm responsible for that.
        JoeAltmaier 12y ago
        Currently there's no requirement in law to be a hero. Some famous court case about a taxi driver co-opted by a bank robber or some such. Autonomous car algorithms must face this issue immediately. Choosing to serve the driver/owner's interest will be a compelling sales point.
chrisBob 12y ago
There is always a balance. At some point these types of algorithms will probably be regulated, and that will take some of the pressure off of both owners and manufacturers. Given the choice I would usually pick me over you, but I would also be ok with buying a car that would choose a tree over a group of kids. This is especially true given that the autonomous car is safer for me overall, even if it takes that one split-second choice away.
muaddirac 12y ago
In the long term, if all cars are autonomous and communicating, it's reasonable to think that most situations could be handled by carefully coordinated group action so that no one is injured. (Especially in the example given in the article - if only one car has a problem, the others should compensate.) In the near term, though? For legal reasons it might be necessary to hand control immediately to a human driver to let them make the decision. Or perhaps far, far more stringent inspection requirements could help mitigate some of this?
- JoeAltmaier 12y ago
  If civil-rights-conscious Americans demand the right to manually control their cars, then its possible an unpredicted human operation (abrupt stop/turn) could upset the autonomous grid and create unsolvable problems - cars going too fast to adapt in time. Either the entire grid would have to slow down enough for any unanticipated human action (which are unlimited in variety), or they would have to 'push the envelope' and accommodate only the likely behaviors. This could very likely result in facing the issue raised by OP: does the autonomous car(s) sacrifice itself, or hit the human driver while preemptively engaging safety devices for its own driver? The designers of the autonomous network will have to face this issue immediately. Its not some remote possibility; its part of the very fabric of the network.
johngalt 12y ago
This article goes out of it's way to create hand wringing 'what-if' scenarios. Completely ignoring the fact that car accidents would be practically eliminated with autonomous vehicles. It's perfectly reasonable for a group of cars to all take action with their owners self-preservation in mind first and foremost. If your worried about 'greater total number of deaths', look at human drivers.
- thematt 12y ago
  I still think it's an interesting question to think about. There's going to be a (long) period of time whereby autonomous vehicles are sharing the road with human controlled vehicles and the elimination of all/most accidents is not likely during that transition.

24 points by maxcan 12y ago

serf 12y ago

now there's a selling point. "Hyper-aggressive BMW AI takes out 15 pedestrians and causes $250,000 in collateral city-wide damage. Reason cited : danger to owner's career prospects from possible late business lunch arrival. Thankfully the owner is reported to be safe (and punctual)."

smrtinsert 12y ago

Yes, and knowing that, I'm sure everyone will buy an autonomous car, so that it will save the many, and not their family.

EpicEng 12y ago
I don't know why this is getting downvoted. Sorry, but our most basic human instinct is that of survival. By that I mean my and my family's survival, not yours. I'm not saying that I wouldn't sacrifice myself to save many other people (no one really knows what they would do until they actually have to make that call), but I'm not going to buy something that will sacrifice my wife and child because a computer thinks it's the best outcome.
- fivethree 12y ago
  Congrats, you've exposed the failings of human logic and ethics!

dllthomas 12y ago

It seems to me that - socially - we should regard this as the responsibility of the driver rather than the automaker. They chose to drive in that place at that time, they maintain their vehicle, and they have chosen some balance between safety and heroism (either by tweaking a setting or by picking what car they buy). It's the driver's agency at work.

krapp 12y ago
At first, sure. But the goal of autonomous cars is to eventually remove driver agency altogether. What happens when all cars are fully autonomous? Will they collectively weight their passengers by "killability", perhaps based on age, insurance premium, marital status, social media status, etc?
- dllthomas 12y ago
  I don't follow. You're proposing there will be driverless cars pursuing their own agenda, as opposed to the agenda of some human or group of humans who have - in some sense - deployed them? That sounds unlikely before the robot uprising.
  - krapp 12y ago
    I don't know about "pursuing their own agenda" but making their own decisions - yes, probably. There isn't much point to autonomous cars which are essentially drones.
    - dllthomas 12y ago
      Decisions don't create agency. My router makes decisions about what packets to route (and where) and what packets to drop. It does so based on a policy I set and those decisions are an extension of my agency. If I configure my car to prefer saving me to saving 100 schoolchildren (or pick a car with that configuration over the alternative), I'm responsible for that.
      - JoeAltmaier 12y ago
        Currently there's no requirement in law to be a hero. Some famous court case about a taxi driver co-opted by a bank robber or some such. Autonomous car algorithms must face this issue immediately. Choosing to serve the driver/owner's interest will be a compelling sales point.

chrisBob 12y ago

There is always a balance. At some point these types of algorithms will probably be regulated, and that will take some of the pressure off of both owners and manufacturers. Given the choice I would usually pick me over you, but I would also be ok with buying a car that would choose a tree over a group of kids. This is especially true given that the autonomous car is safer for me overall, even if it takes that one split-second choice away.

muaddirac 12y ago

In the long term, if all cars are autonomous and communicating, it's reasonable to think that most situations could be handled by carefully coordinated group action so that no one is injured. (Especially in the example given in the article - if only one car has a problem, the others should compensate.) In the near term, though? For legal reasons it might be necessary to hand control immediately to a human driver to let them make the decision. Or perhaps far, far more stringent inspection requirements could help mitigate some of this?

JoeAltmaier 12y ago
If civil-rights-conscious Americans demand the right to manually control their cars, then its possible an unpredicted human operation (abrupt stop/turn) could upset the autonomous grid and create unsolvable problems - cars going too fast to adapt in time. Either the entire grid would have to slow down enough for any unanticipated human action (which are unlimited in variety), or they would have to 'push the envelope' and accommodate only the likely behaviors. This could very likely result in facing the issue raised by OP: does the autonomous car(s) sacrifice itself, or hit the human driver while preemptively engaging safety devices for its own driver? The designers of the autonomous network will have to face this issue immediately. Its not some remote possibility; its part of the very fabric of the network.

johngalt 12y ago

This article goes out of it's way to create hand wringing 'what-if' scenarios. Completely ignoring the fact that car accidents would be practically eliminated with autonomous vehicles. It's perfectly reasonable for a group of cars to all take action with their owners self-preservation in mind first and foremost. If your worried about 'greater total number of deaths', look at human drivers.

thematt 12y ago
I still think it's an interesting question to think about. There's going to be a (long) period of time whereby autonomous vehicles are sharing the road with human controlled vehicles and the elimination of all/most accidents is not likely during that transition.