4 ms·

Every time I read one of these articles I feel like the author doesn't have a very good grasp of exactly how these work. The car doesn't have to "choose". Nobod

Every time I read one of these articles I feel like the author doesn't have a very good grasp of exactly how these work. The car doesn't have to "choose". Nobody at Google is programming these things to calculate the life value of humans, the percent chance of injury to occupants, or the benefits of sacrificing pedestrians. They're programming them to try and avoid accidents in general. There will be times when the computer fails to do that, but that doesn't mean it decided to kill someone by making a moral choice about human life priority. Maybe in the future this will be a relevant argument but the sort of algorithms mentioned that decide which crash scenario is 'best' simply don't exist. The best way to deal with these moral issues is to just... not program that behaviour.

m0nty 11y ago
I think people are excited about this because it seems to mesh so closely to the Trolley Problem, which has been an important thought experiment in the study of ethics. https://en.wikipedia.org/wiki/Trolley_problem https://en.wikipedia.org/wiki/Trolley_problem (They even have an "Implications for autonomous vehicles" section now.) So, who knew? Suddenly it's not so theoretical, it apparently has a real-world application. Except, as you seem to be getting at, probably the only thing engineers will be considering in any depth is how and when to brake, not trying to swerve the car elaborately in one direction or the other. The number of occasions when a serious accident occurs because a driver reflexively swerved to avoid an animal or other obstruction (sometimes literally dozens dead in one accident) emphasises that swerving is not so good a strategy as braking. Or, as my driving instructor once told me, "There is only one cause of accidents on our roads: failing to stop in time."
- Already__Taken 11y ago
  A car has a fixed contact patch with the road, 100% of this is used to accelerate, brake, or turn. Any combination of those must stay within that 100%. Why would a computer choose to swerve and not slam the brakes. With reactions that are as fast as a computer could be what kind of a situation even is a swerve vs a course correction.
adventured 11y ago
Even in the near future, I'm skeptical a car's AI is going to be able to know how many babies are in the other car, such that it decides to kill me instead of putting said babies at risk. It's such a reach as to be silly. Then if you run the outlandish scenario, and assume extraordinary leaps in AI comprehension in the next 30 or 40 years, it still remains silly. How many teenagers are equal to one baby? How many puppies does it take to equal one 80 year old grinch? There is no scenario under which it will work. Ultimately the AI will not be able to make accurate judgments about morality / character, it can never choose correctly even if you try to program it with what would be strictly subjective value systems - that one flaw guarantees it can never work properly.
- TeMPOraL 11y ago
  > How many teenagers are equal to one baby? How many puppies does it take to equal one 80 year old grinch? There is no scenario under which it will work. Well, this is something we don't expect from humans. No human, except maybe high-ranking businessmen, leaders and armchair ethicists ever does that kind of calculation, because in the usual case we meet them, there is no time. When you're a second before a car crash, you don't count the passengers of the other vehicle and evaluate their relative worth compared to yourself. You swerve, or brake, or generally do whatever you can to not hit them. And so will the self-driving cars, except better. In fact, I think we can avoid the moral issue altogether by just focusing on making it nigh-impossible to surprise an autonomous vehicle in such a way that it won't be able to reach a safe state[0] in time. After we have that, then we may consider optimizing the algorithms to take into account that e.g. the other vehicle is empty, so its suitable to be used as a braking device. [0] - Either avoidance trajectory or a speed that would not be likely to kill a pedestrian if they were hit directly.
archgoon 11y ago
> Nobody at Google is programming these things to calculate the life value of humans. Well, not everyone at Google agrees with you. Yonatan Zunger seems to think that yes, you do have to work out what the car has to do in certain worst case scenarios. 'A self-driving car, on the other hand, would require that these decisions be specified in advance.' https://plus.google.com/+YonatanZunger/posts/FMdAD6ckH38 https://plus.google.com/+YonatanZunger/posts/FMdAD6ckH38 And well, he's right. Once you have programmed the car, it will respond deterministically to inputs. You can close your eyes and ignore the consequences of the decisions you made for the Trolley analogs, but that's willful ignorance of your decisions at that point (and bad QA).
- chrisfosterelli 11y ago
  > And well, he's right. Once you have programmed the car, it will respond deterministically to inputs. This is true, but that doesn't mean that the response is based on the car making moral decisions about the value of the human lives involved. As the commenters below mentioned, the technology doesn't even exist at this point to do anything other than avoid the object in front of you and try not to crash while doing it. Sometimes this may result in a dead pedestrian and sometimes this may result in a dead occupant. That isn't the car making a moral judgement, it's just collision avoidance.
  - coldtea 11y ago
    It doesn't have to be a moral judgement on behalf of the car -- after all it's just running a collision avoidance algorithm. But the human designers of the "collision avoidance" algorithm will design it with moral judgements in mind. E.g. what to do when face recognition shows 10 kids in from of you and you have 1 passenger in the car that if you turn you'll avoid hitting them but risk his life.
wysiwylwysiwyl 11y ago
The algorithm faces a choice when a drunk driver is on collision course and evasive action is required, what if there is no safe area to steer towards? It is a choice by design whether the algorithm deals with this situation by inactivity, failure or selecting the scenario with the lowest collision speed.
coldtea 11y ago
>Every time I read one of these articles I feel like the author doesn't have a very good grasp of exactly how these work. The car doesn't have to "choose". Nobody at Google is programming these things to calculate the life value of humans, the percent chance of injury to occupants, or the benefits of sacrificing pedestrians. You'd be surprised. Actually even if they don't, they can (and probably will) be regulated to do so.
alephu5 11y ago
Moral issues aside, don't you think that crashing well is an important design feature? For example, to use ultrasound to map the hardness of nearby obstacles and go for the one that provides the quickest safe deceleration, or to orchestrate a decelerating glance with an oncoming vehicle?

m0nty 11y ago

I think people are excited about this because it seems to mesh so closely to the Trolley Problem, which has been an important thought experiment in the study of ethics. https://en.wikipedia.org/wiki/Trolley_problem https://en.wikipedia.org/wiki/Trolley_problem (They even have an "Implications for autonomous vehicles" section now.) So, who knew? Suddenly it's not so theoretical, it apparently has a real-world application. Except, as you seem to be getting at, probably the only thing engineers will be considering in any depth is how and when to brake, not trying to swerve the car elaborately in one direction or the other. The number of occasions when a serious accident occurs because a driver reflexively swerved to avoid an animal or other obstruction (sometimes literally dozens dead in one accident) emphasises that swerving is not so good a strategy as braking. Or, as my driving instructor once told me, "There is only one cause of accidents on our roads: failing to stop in time."

Already__Taken 11y ago
A car has a fixed contact patch with the road, 100% of this is used to accelerate, brake, or turn. Any combination of those must stay within that 100%. Why would a computer choose to swerve and not slam the brakes. With reactions that are as fast as a computer could be what kind of a situation even is a swerve vs a course correction.

adventured 11y ago

Even in the near future, I'm skeptical a car's AI is going to be able to know how many babies are in the other car, such that it decides to kill me instead of putting said babies at risk. It's such a reach as to be silly. Then if you run the outlandish scenario, and assume extraordinary leaps in AI comprehension in the next 30 or 40 years, it still remains silly. How many teenagers are equal to one baby? How many puppies does it take to equal one 80 year old grinch? There is no scenario under which it will work. Ultimately the AI will not be able to make accurate judgments about morality / character, it can never choose correctly even if you try to program it with what would be strictly subjective value systems - that one flaw guarantees it can never work properly.

TeMPOraL 11y ago
> How many teenagers are equal to one baby? How many puppies does it take to equal one 80 year old grinch? There is no scenario under which it will work. Well, this is something we don't expect from humans. No human, except maybe high-ranking businessmen, leaders and armchair ethicists ever does that kind of calculation, because in the usual case we meet them, there is no time. When you're a second before a car crash, you don't count the passengers of the other vehicle and evaluate their relative worth compared to yourself. You swerve, or brake, or generally do whatever you can to not hit them. And so will the self-driving cars, except better. In fact, I think we can avoid the moral issue altogether by just focusing on making it nigh-impossible to surprise an autonomous vehicle in such a way that it won't be able to reach a safe state[0] in time. After we have that, then we may consider optimizing the algorithms to take into account that e.g. the other vehicle is empty, so its suitable to be used as a braking device. [0] - Either avoidance trajectory or a speed that would not be likely to kill a pedestrian if they were hit directly.

archgoon 11y ago

> Nobody at Google is programming these things to calculate the life value of humans. Well, not everyone at Google agrees with you. Yonatan Zunger seems to think that yes, you do have to work out what the car has to do in certain worst case scenarios. 'A self-driving car, on the other hand, would require that these decisions be specified in advance.' https://plus.google.com/+YonatanZunger/posts/FMdAD6ckH38 https://plus.google.com/+YonatanZunger/posts/FMdAD6ckH38 And well, he's right. Once you have programmed the car, it will respond deterministically to inputs. You can close your eyes and ignore the consequences of the decisions you made for the Trolley analogs, but that's willful ignorance of your decisions at that point (and bad QA).

chrisfosterelli 11y ago
> And well, he's right. Once you have programmed the car, it will respond deterministically to inputs. This is true, but that doesn't mean that the response is based on the car making moral decisions about the value of the human lives involved. As the commenters below mentioned, the technology doesn't even exist at this point to do anything other than avoid the object in front of you and try not to crash while doing it. Sometimes this may result in a dead pedestrian and sometimes this may result in a dead occupant. That isn't the car making a moral judgement, it's just collision avoidance.
- coldtea 11y ago
  It doesn't have to be a moral judgement on behalf of the car -- after all it's just running a collision avoidance algorithm. But the human designers of the "collision avoidance" algorithm will design it with moral judgements in mind. E.g. what to do when face recognition shows 10 kids in from of you and you have 1 passenger in the car that if you turn you'll avoid hitting them but risk his life.

wysiwylwysiwyl 11y ago

The algorithm faces a choice when a drunk driver is on collision course and evasive action is required, what if there is no safe area to steer towards? It is a choice by design whether the algorithm deals with this situation by inactivity, failure or selecting the scenario with the lowest collision speed.

coldtea 11y ago

>Every time I read one of these articles I feel like the author doesn't have a very good grasp of exactly how these work. The car doesn't have to "choose". Nobody at Google is programming these things to calculate the life value of humans, the percent chance of injury to occupants, or the benefits of sacrificing pedestrians. You'd be surprised. Actually even if they don't, they can (and probably will) be regulated to do so.

alephu5 11y ago

Moral issues aside, don't you think that crashing well is an important design feature? For example, to use ultrasound to map the hardness of nearby obstacles and go for the one that provides the quickest safe deceleration, or to orchestrate a decelerating glance with an oncoming vehicle?