5 ms·

> Should the self driving car swerve into you to avoid greater moral hazard? The self-driving car's first priority should be to protect those inside it. Anythi

> Should the self driving car swerve into you to avoid greater moral hazard? The self-driving car's first priority should be to protect those inside it. Anything else is more complicated. I read an article a little while ago about this topic and it posed a question along the lines of "should a self-driving car do the thing that'll kill its own passengers, yet save a larger number of others around it?" Think about how difficult it would be for a self-driving car to estimate this. Even more importantly, how would two self-driving cars cooperate in the event of an inevitable accident? Would we allow them to directly talk to each other, or are they only allowed to interpret each others' trajectories? I think the simplest directive is "save your own passengers". That is what human drivers currently do, and I believe in the long run this would be effective at saving the optimal number of lives. Besides, if choosing between a car with this directive and a car that's programmed to sacrifice you to save someone else, which would you buy?

anthony_d 11y ago
I'm not sure about that. If your car had uncontrolled acceleration, for instance, would you consider use a crowd of people to slow you down or would you put it in the ditch? For the sake of argument that amounts to a left or right turn.
- deleted 11y ago
  [deleted]
- IgorPartola 11y ago
  Say there are 10 people in this crowd. The optimal solution for the 11 of us is obviously to have my car drive off a cliff to the right, not into the crowd on the left. However, for me, the optimal solution is obviously to not drive off a cliff. However, your argument omits a couple of interesting details. First off, if a car can have uncontrolled acceleration, it can likely have any number of glitches. It may for example think it's accelerating uncontrollably, and try to throw me off a cliff. Or it may think that the cliff is a small ditch. Or it may think that a bunch of balloons tied to a mail box is a crowd. If you are a programmer, what would you rather debug: code that prevents uncontrolled acceleration or code responsible for killing the driver by recognizing crowds and cliffs? The other details is whether the metaphorical crowd should even allow me to buy a car that has logic built into it to drive over them. What if instead of a cliff and a crowd it's actually two different crowds of different sizes? What would a human driver do in these cases?
  - anthony_d 11y ago
    As a person and a driver I'm willing to say the car should drive off the cliff. As a developer I wouldn't buy a car that would drive off the cliff. It's a tough problem. Personally I would make a quickly/poorly thought out analysis of how to do the least damage. Being human I'd probably get it wrong.
    - IgorPartola 11y ago
      I am more or less in the same boat as you. I do however think that a simple algorithm of "save people in the car" is better than "try to save objects identified as people outside the car" because it is simple and in the long run likely very effective. Making computers make moral/ethical decisions is difficult, especially with poor inputs.
    - bladecatcher 11y ago
      >> As a developer I wouldn't buy a car that would drive off the cliff. What does being a developer have anything to do with it?
      - grkvlt 11y ago
        A developer knows, through hard-won experience, that the algorithm to decide whether driving over a cliff is better for everyone is bound to have bugs, and will therefore gladly throw you off a cliff when it is the wrong thing to do ;( Non-developers would assume the computer to be (somewhat) infallible.
  - maxerickson 11y ago
    Humans typically avoid creating the choice between crowds to drive into (by following traffic laws, slowing down when there are obstructions in the street, etc.). I haven't tried to figure out what the numbers are, but my gut expectation is that vehicles driving into crowds is so rare that there aren't really statistics about it.
- e12e 11y ago
  I think a more realistic moral scenario is this: you come around a bend, and there's a five year old on a tricycle in the road. He shouldn't be there; but he is. Do you plow over him, with minimal damage to the car, or try to break/avoid him with the risk of going off the road and into the ocean below - or hitting the cliff? What if you're in your car with your family? The best answer is of course: never let the car travel so fast this scenario happens - but given the possibility of an oil spill around the bend -- that might be very slow indeed.
  - vegedor 11y ago
    This is a tough question because it's emotionally loaded. What if it was a drunk bum instead of a toddler evoking protective feelings? Unless the algorithms are good enough to evade without running off the cliff, it's not a clear cut case. If you look at how bureaucracy works, that is largely automated or at least rule based, and how a statistic minority gets run over by unfair legislation, i'm sure the toddler wouldn't have the slightest chance. From a dystopian perspective, insurance companies would calculate in split second, who's valued higher.
cortesoft 11y ago
Eh, I don't know. Lots of people would swerve their car into a tree to avoid hitting a person. They put themselves at risk to avoid killing someone.
- hamitron 11y ago
  I think this is because I would feel responsible for killing that person. If my self-driving car hit the person, I may feel bad, but probably not responsible.
malkia 11y ago
Most-obvious, but a self-driving would be riding "alone" for long periods of time. For example "sending" car from one member of the family to another (or one client to another for use), or maybe delivering items from the store directly into a box. So in this case, the car should have the other mode, which is not safe for the car, but from whatever is outside, but now that I've typed it, even that I don't know the details, I feel that's probably too complicated.
- IgorPartola 11y ago
  Ooh, I really like this idea. Instead of me finding parking at the airport, I send my car to park in my own driveway, then have it pick me up a few days later. That's brilliant. I haven't even thought of the possibility of having the car drive empty.
  - ArtDev 11y ago
    It is amazing when you think about it at scale. Your car drops you off exactly where you want to go, then double-parks in a ultra-dense parking hanger. What are we going to do with all the extra parking spaces? Turn them into parks/wildlife habitat/green space!
    - IgorPartola 11y ago
      To be fair, we are probably going to just keep them as parking lots and pile more people into the cities. But yes, this type of thing is exciting.
  - chinpokomon 11y ago
    This has been discussed as the first step for wide acceptance of driverless cars. As a personal valet at a mall for instance, large public lots could be prepared in a way that makes navigation for driverless cars easier. There could be a driverless car parking lot away from main entrances, and therefore leave most parking open for regular drivers. I believe it was Mercedes that had an interesting demo a year or two ago in Las Vegas that demonstrated this sort of idea.
lotu 11y ago
Realistically the self-driving car isn't going to be smart or fast enough to make these choices. We are assuming some infallible perfect AI when if-fact you have a bunch of (relatively) simple algorithms a bunch of mapping data. The fact is that the type of complicated moral analysis will take time prehaps in the absolute best case several 1/10th of a second. For a moving vehicle that analysis will likely be moot during that time. We don't expect human drivers to make such difficulty snap decisions as part of a driving tests why should we ask a self driving car to?
rralian 11y ago
Really fascinating questions. > Think about how difficult it would be for a self-driving car to estimate this. For sure you're right in the near term. But assuming we achieve super-intelligent AI that we manage to corral into a benevolent relationship with us, perhaps difficulty wouldn't be an issue at that point. > Even more importantly, how would two self-driving cars cooperate in the event of an inevitable accident? Would we allow them to directly talk to each other, or are they only allowed to interpret each others' trajectories? Yeah, this is getting good. Assuming we have those super-intelligent machines, those cars should be able to share pretty much all relevant data before a collision. So how could the car determine the greater good option? Is one life equal to another? What if one is rich and one is poor? What if one is the president? How could they even collaborate if they don't share the same basis for valuing human life? Could car companies realistically have diversity in those algorithms, or wouldn't that just lead to lawsuits against cars that were overly aggressive or passive? Nobody wants an unsafe car, so would car companies have to compete in their creation of aggressive algorithms? Or would it be a factor of cost... that you need to pay more for a car that guards its occupants more strongly... leading to more and more aggressive cars as they compete for the crown? Honestly, this line of thinking leads me to think we'll eventually need these values to be dictated and homogenized... requiring a strong central government, and aggressive detection and punishment of aberration. Or else a mad max style zero sum game. Forgive the digression into fantasyland, I just found the ideas and questions here to be inspiringly interesting.
fapjacks 11y ago
These are some interesting points, but we can look for a peek into the future with airplanes which communicate through anticollision computer systems. I'm not an aviation person, but I believe the actual collision avoidance must be performed by a human being, however the devices themselves communicate with one another to determine the corrective flight path. Also, you can enforce cars with directives to save "the most" lives with policy and law. Of course people are going to buy the crowd-plowing car if it saves their own asses. That's why you make it difficult or impossible to buy a car that will do this, instead only authorizing cars that make the better choice to drive the streets.
- aeyler 11y ago
  I've worked on TCAS (collision avoidance for airplanes) in the past. The system will alert the pilot and ask them to do a specific route (such as climb, descend, do-not-climb). It's up to the pilot to respond (est. 2-5 seconds reaction time). The route decided upon by the airplane is in conjunction with the other plane.
  - fapjacks 11y ago
    Thank you for your input! I'm glad memory serves me mostly correctly every once in a great while. Will the TCAS or a related system take corrective action if the pilot does not?
chinpokomon 11y ago
What is the ethical answer for a human in the same situation?
Schwolop 11y ago
Respectfully disagree. In a region of inevitable collision, the car's first priority should be to reduce its kinetic energy to zero. If you take that dictum, then any action which requires activation energy (e.g. steering, or accelerating to avoid something) is unavailable, and as a result, the vast majority of these ethical decisions are completely moot.

anthony_d 11y ago

I'm not sure about that. If your car had uncontrolled acceleration, for instance, would you consider use a crowd of people to slow you down or would you put it in the ditch? For the sake of argument that amounts to a left or right turn.

deleted 11y ago
[deleted]
IgorPartola 11y ago
Say there are 10 people in this crowd. The optimal solution for the 11 of us is obviously to have my car drive off a cliff to the right, not into the crowd on the left. However, for me, the optimal solution is obviously to not drive off a cliff. However, your argument omits a couple of interesting details. First off, if a car can have uncontrolled acceleration, it can likely have any number of glitches. It may for example think it's accelerating uncontrollably, and try to throw me off a cliff. Or it may think that the cliff is a small ditch. Or it may think that a bunch of balloons tied to a mail box is a crowd. If you are a programmer, what would you rather debug: code that prevents uncontrolled acceleration or code responsible for killing the driver by recognizing crowds and cliffs? The other details is whether the metaphorical crowd should even allow me to buy a car that has logic built into it to drive over them. What if instead of a cliff and a crowd it's actually two different crowds of different sizes? What would a human driver do in these cases?
- anthony_d 11y ago
  As a person and a driver I'm willing to say the car should drive off the cliff. As a developer I wouldn't buy a car that would drive off the cliff. It's a tough problem. Personally I would make a quickly/poorly thought out analysis of how to do the least damage. Being human I'd probably get it wrong.
  - IgorPartola 11y ago
    I am more or less in the same boat as you. I do however think that a simple algorithm of "save people in the car" is better than "try to save objects identified as people outside the car" because it is simple and in the long run likely very effective. Making computers make moral/ethical decisions is difficult, especially with poor inputs.
  - bladecatcher 11y ago
    >> As a developer I wouldn't buy a car that would drive off the cliff. What does being a developer have anything to do with it?
    - grkvlt 11y ago
      A developer knows, through hard-won experience, that the algorithm to decide whether driving over a cliff is better for everyone is bound to have bugs, and will therefore gladly throw you off a cliff when it is the wrong thing to do ;( Non-developers would assume the computer to be (somewhat) infallible.
- maxerickson 11y ago
  Humans typically avoid creating the choice between crowds to drive into (by following traffic laws, slowing down when there are obstructions in the street, etc.). I haven't tried to figure out what the numbers are, but my gut expectation is that vehicles driving into crowds is so rare that there aren't really statistics about it.
e12e 11y ago
I think a more realistic moral scenario is this: you come around a bend, and there's a five year old on a tricycle in the road. He shouldn't be there; but he is. Do you plow over him, with minimal damage to the car, or try to break/avoid him with the risk of going off the road and into the ocean below - or hitting the cliff? What if you're in your car with your family? The best answer is of course: never let the car travel so fast this scenario happens - but given the possibility of an oil spill around the bend -- that might be very slow indeed.
- vegedor 11y ago
  This is a tough question because it's emotionally loaded. What if it was a drunk bum instead of a toddler evoking protective feelings? Unless the algorithms are good enough to evade without running off the cliff, it's not a clear cut case. If you look at how bureaucracy works, that is largely automated or at least rule based, and how a statistic minority gets run over by unfair legislation, i'm sure the toddler wouldn't have the slightest chance. From a dystopian perspective, insurance companies would calculate in split second, who's valued higher.

cortesoft 11y ago

Eh, I don't know. Lots of people would swerve their car into a tree to avoid hitting a person. They put themselves at risk to avoid killing someone.

hamitron 11y ago
I think this is because I would feel responsible for killing that person. If my self-driving car hit the person, I may feel bad, but probably not responsible.

malkia 11y ago

Most-obvious, but a self-driving would be riding "alone" for long periods of time. For example "sending" car from one member of the family to another (or one client to another for use), or maybe delivering items from the store directly into a box. So in this case, the car should have the other mode, which is not safe for the car, but from whatever is outside, but now that I've typed it, even that I don't know the details, I feel that's probably too complicated.

IgorPartola 11y ago
Ooh, I really like this idea. Instead of me finding parking at the airport, I send my car to park in my own driveway, then have it pick me up a few days later. That's brilliant. I haven't even thought of the possibility of having the car drive empty.
- ArtDev 11y ago
  It is amazing when you think about it at scale. Your car drops you off exactly where you want to go, then double-parks in a ultra-dense parking hanger. What are we going to do with all the extra parking spaces? Turn them into parks/wildlife habitat/green space!
  - IgorPartola 11y ago
    To be fair, we are probably going to just keep them as parking lots and pile more people into the cities. But yes, this type of thing is exciting.
- chinpokomon 11y ago
  This has been discussed as the first step for wide acceptance of driverless cars. As a personal valet at a mall for instance, large public lots could be prepared in a way that makes navigation for driverless cars easier. There could be a driverless car parking lot away from main entrances, and therefore leave most parking open for regular drivers. I believe it was Mercedes that had an interesting demo a year or two ago in Las Vegas that demonstrated this sort of idea.

lotu 11y ago

Realistically the self-driving car isn't going to be smart or fast enough to make these choices. We are assuming some infallible perfect AI when if-fact you have a bunch of (relatively) simple algorithms a bunch of mapping data. The fact is that the type of complicated moral analysis will take time prehaps in the absolute best case several 1/10th of a second. For a moving vehicle that analysis will likely be moot during that time. We don't expect human drivers to make such difficulty snap decisions as part of a driving tests why should we ask a self driving car to?

rralian 11y ago

Really fascinating questions. > Think about how difficult it would be for a self-driving car to estimate this. For sure you're right in the near term. But assuming we achieve super-intelligent AI that we manage to corral into a benevolent relationship with us, perhaps difficulty wouldn't be an issue at that point. > Even more importantly, how would two self-driving cars cooperate in the event of an inevitable accident? Would we allow them to directly talk to each other, or are they only allowed to interpret each others' trajectories? Yeah, this is getting good. Assuming we have those super-intelligent machines, those cars should be able to share pretty much all relevant data before a collision. So how could the car determine the greater good option? Is one life equal to another? What if one is rich and one is poor? What if one is the president? How could they even collaborate if they don't share the same basis for valuing human life? Could car companies realistically have diversity in those algorithms, or wouldn't that just lead to lawsuits against cars that were overly aggressive or passive? Nobody wants an unsafe car, so would car companies have to compete in their creation of aggressive algorithms? Or would it be a factor of cost... that you need to pay more for a car that guards its occupants more strongly... leading to more and more aggressive cars as they compete for the crown? Honestly, this line of thinking leads me to think we'll eventually need these values to be dictated and homogenized... requiring a strong central government, and aggressive detection and punishment of aberration. Or else a mad max style zero sum game. Forgive the digression into fantasyland, I just found the ideas and questions here to be inspiringly interesting.

fapjacks 11y ago

These are some interesting points, but we can look for a peek into the future with airplanes which communicate through anticollision computer systems. I'm not an aviation person, but I believe the actual collision avoidance must be performed by a human being, however the devices themselves communicate with one another to determine the corrective flight path. Also, you can enforce cars with directives to save "the most" lives with policy and law. Of course people are going to buy the crowd-plowing car if it saves their own asses. That's why you make it difficult or impossible to buy a car that will do this, instead only authorizing cars that make the better choice to drive the streets.

aeyler 11y ago
I've worked on TCAS (collision avoidance for airplanes) in the past. The system will alert the pilot and ask them to do a specific route (such as climb, descend, do-not-climb). It's up to the pilot to respond (est. 2-5 seconds reaction time). The route decided upon by the airplane is in conjunction with the other plane.
- fapjacks 11y ago
  Thank you for your input! I'm glad memory serves me mostly correctly every once in a great while. Will the TCAS or a related system take corrective action if the pilot does not?

chinpokomon 11y ago

What is the ethical answer for a human in the same situation?

Schwolop 11y ago

Respectfully disagree. In a region of inevitable collision, the car's first priority should be to reduce its kinetic energy to zero. If you take that dictum, then any action which requires activation energy (e.g. steering, or accelerating to avoid something) is unavailable, and as a result, the vast majority of these ethical decisions are completely moot.