4 ms·

This is really impressive. However, has any one else thought about the ethical dilemmas these algorithms will have to decide upon? Say for instance that two pe

This is really impressive. However, has any one else thought about the ethical dilemmas these algorithms will have to decide upon? Say for instance that two pedestrians, a young child and an old person, suddenly find themselves in front of a self-driving car. There is not time for the car to brake in time so the algorithm has to make a choice: Which pedestrian gets hit? I think we'll see more questions like these in the coming years as AI progresses and we become more dependent on it.

Nadya 11y ago
This assumes a flaw in the cars algorithm to be travelling too quickly at an upcoming intersection - or a pedestrian breaking the law by jay-walking or crossing during a "no cross" time. Let's give it the most realistic scenario of a blind-turn intersection with an obstruction such as a bush make it impossible to see the sidewalk on the right-hand side. The two pedestrians in question are a grandparent and their grandchild crossing without looking both ways for their safety. Is there oncoming traffic? Is there an empty sidewalk to drive on? Can the car make an attempt to brake as safely and quickly as possible and sound the horn to alert the pedestrians to get out of the way? Is it safe enough to perform a handbrake turn? Is it safe enough to perform a bootleg turn? Before asking which pedestrian the car chooses to hit, I would exhaust all other available options to prevent such a scenario from happening in the first place.
- michaelmcmillan 11y ago
  So your answer to my hypothetical question of what an algorithm should do in a situation where whatever the outcome a life will be lost, is to say that this scenario will never happen without a flaw in the cars algorithm? That seems very narrow minded to me. Let me put it this way: When all your other available options are exhausted what should the choice be?
  - Nadya 11y ago
    There has to be too little room to perform an evasive maneuver. The road has to be too crowded or too small to perform an evasive maneuver. The pedestrians would have to be deaf or suffer from "deer fright". The car would have to be travelling too quickly to brake and would have had to fail to notice the pedestrians about to cross. The scenario is one that requires so many worst-case-scenario events to be happening simultaneously as to say the scenario is likely avoidable altogether in one of numerous ways. So yes, I'm calling your hypothetical scenario a little contrived. To answer the hypothetical with the answer you're expecting; people have already made this choice in the past for disaster scenarios: "Women and children first." The algorithm would choose to hit the elderly person over the child in your scenario. The ethical choice becomes more complex if the pedestrians in question are both children or both women. I don't think humans have made an ethical decision on such a choice.
    - michaelmcmillan 11y ago
      So yes, I'm calling your hypothetical scenario a little contrived. Yes, that is the often the point of hypothetical scenarios. I was not expecting any particular answer, I simply find the problem interesting. It begs the question: Does this mean there is a line of code somewhere that ultimately has to make that decision?
      - Nadya 11y ago
        I don't see how that begs the question at all. If an algorithm exists to make such a decision it is a given that the code would exist to allow it to make that decision. Without the code a more simple scenario plays out. The car continues along its current trajectory and kills whichever pedestrian was in front of the vehicle, which could be both of them.
    - zanny 11y ago
      I personally would rather not have sexist cars that would hit a same-aged man over a same-aged woman. Women and children first is not a policy of who is most valuable. It is a policy of who is most vulnerable. In disasters it is presumed that the women and children cannot fend for themselves (and while there is a biological aspect to women being on average weaker than men the presumption of women and children first is still painfully sexist) and that the men coming up behind are more likely to survive than any alternative, where men go first and "the vulnerable" take up the rear. Even besides that, how do you know if a child is more valuable than an elderly person? Its not as simple a problem as yesteryears well intentioned sexism and ageism. But the real answer is much simpler: The car would, in the situation where it could not avoid hitting a person, hit whomever it computes to be most likely to survive.
ufmace 11y ago
I think the real answer to this sort of question is: Who cares? There are tens of thousands of people dying every year in very real car crashes, plus probably an order of magnitude more suffering life-changing injuries, and self-driving cars have a huge potential to cut way down on that. Meanwhile, situations like that virtually never happen - maybe like once a year in the entire US. If they can eliminate even 10% of the actual accidents, then they could run down both pedestrians in that imaginary scenario, and still be way ahead. It may be fun to think about ethical dilemmas like that, but they are fantastically rare compared to the huge numbers of perfectly ordinary crashes that happen every day. Let's fix those, and worry about the rare one-offs after we've reduced the accident rate by 95%.
- michaelmcmillan 11y ago
  Yes, I agree with you. These edge cases will probably never actually happen. But again, this is only hypothetical. Are there lines of code that actually define the behaviour of the car in these cases? I think it's an interesting question (-:

Nadya 11y ago

This assumes a flaw in the cars algorithm to be travelling too quickly at an upcoming intersection - or a pedestrian breaking the law by jay-walking or crossing during a "no cross" time. Let's give it the most realistic scenario of a blind-turn intersection with an obstruction such as a bush make it impossible to see the sidewalk on the right-hand side. The two pedestrians in question are a grandparent and their grandchild crossing without looking both ways for their safety. Is there oncoming traffic? Is there an empty sidewalk to drive on? Can the car make an attempt to brake as safely and quickly as possible and sound the horn to alert the pedestrians to get out of the way? Is it safe enough to perform a handbrake turn? Is it safe enough to perform a bootleg turn? Before asking which pedestrian the car chooses to hit, I would exhaust all other available options to prevent such a scenario from happening in the first place.

michaelmcmillan 11y ago
So your answer to my hypothetical question of what an algorithm should do in a situation where whatever the outcome a life will be lost, is to say that this scenario will never happen without a flaw in the cars algorithm? That seems very narrow minded to me. Let me put it this way: When all your other available options are exhausted what should the choice be?
- Nadya 11y ago
  There has to be too little room to perform an evasive maneuver. The road has to be too crowded or too small to perform an evasive maneuver. The pedestrians would have to be deaf or suffer from "deer fright". The car would have to be travelling too quickly to brake and would have had to fail to notice the pedestrians about to cross. The scenario is one that requires so many worst-case-scenario events to be happening simultaneously as to say the scenario is likely avoidable altogether in one of numerous ways. So yes, I'm calling your hypothetical scenario a little contrived. To answer the hypothetical with the answer you're expecting; people have already made this choice in the past for disaster scenarios: "Women and children first." The algorithm would choose to hit the elderly person over the child in your scenario. The ethical choice becomes more complex if the pedestrians in question are both children or both women. I don't think humans have made an ethical decision on such a choice.
  - michaelmcmillan 11y ago
    So yes, I'm calling your hypothetical scenario a little contrived. Yes, that is the often the point of hypothetical scenarios. I was not expecting any particular answer, I simply find the problem interesting. It begs the question: Does this mean there is a line of code somewhere that ultimately has to make that decision?
    - Nadya 11y ago
      I don't see how that begs the question at all. If an algorithm exists to make such a decision it is a given that the code would exist to allow it to make that decision. Without the code a more simple scenario plays out. The car continues along its current trajectory and kills whichever pedestrian was in front of the vehicle, which could be both of them.
  - zanny 11y ago
    I personally would rather not have sexist cars that would hit a same-aged man over a same-aged woman. Women and children first is not a policy of who is most valuable. It is a policy of who is most vulnerable. In disasters it is presumed that the women and children cannot fend for themselves (and while there is a biological aspect to women being on average weaker than men the presumption of women and children first is still painfully sexist) and that the men coming up behind are more likely to survive than any alternative, where men go first and "the vulnerable" take up the rear. Even besides that, how do you know if a child is more valuable than an elderly person? Its not as simple a problem as yesteryears well intentioned sexism and ageism. But the real answer is much simpler: The car would, in the situation where it could not avoid hitting a person, hit whomever it computes to be most likely to survive.

ufmace 11y ago

I think the real answer to this sort of question is: Who cares? There are tens of thousands of people dying every year in very real car crashes, plus probably an order of magnitude more suffering life-changing injuries, and self-driving cars have a huge potential to cut way down on that. Meanwhile, situations like that virtually never happen - maybe like once a year in the entire US. If they can eliminate even 10% of the actual accidents, then they could run down both pedestrians in that imaginary scenario, and still be way ahead. It may be fun to think about ethical dilemmas like that, but they are fantastically rare compared to the huge numbers of perfectly ordinary crashes that happen every day. Let's fix those, and worry about the rare one-offs after we've reduced the accident rate by 95%.

michaelmcmillan 11y ago
Yes, I agree with you. These edge cases will probably never actually happen. But again, this is only hypothetical. Are there lines of code that actually define the behaviour of the car in these cases? I think it's an interesting question (-: