6 ms·

“ if a car detects a sudden obstacle—say, a jackknifed truck—should it hit the truck and kill its own driver, or should it swerve onto a crowded sidewalk and ki

“ if a car detects a sudden obstacle—say, a jackknifed truck—should it hit the truck and kill its own driver, or should it swerve onto a crowded sidewalk and kill pedestrians? A human driver might react randomly (if she has time to react at all), but the response of an autonomous vehicle would have to be programmed ahead of time. ” I hate this logic. No, it didn’t need to be programmed ahead of time. It should try to avoid crashing. It shouldn’t try to solve a philosophical problem. We don’t want to require our cars to observe it’s surroundings and calculate number of things identified as humans to minimize expected loss of life. That’s difficult and error prone. If you’re going down a street and something jumps in your way, make the program try not to hit anything. Ideally by braking, because you shouldn’t be going fast enough to not be able to do so based on your environment.

mr_toad 8y ago
Agreed. Swerving onto sidewalks or deliberately hitting pedestrians or vehicles is illegal anyway. People talking about self driving ethics should spend less time reading philosophy and more time reading the road rules.
gowld 8y ago
And even if a car was implausibl in the same situation with the same options, it still of course could choose a response randomly or arbitrarily based on how the sorting algorithm happened to rank equal outcomes.
pikma 8y ago
I agree that it's mostly a theoretical problem. But in the long term, if I imagine a world where we all cars are self driving, there would still be accidents from time to time, for things that can't be predicted: a rock falls on the road, an earthquake happens and destroys the road, etc. This would be rare, but would still kill a few thousand people every year. Wouldn't it be natural then to tweak the algorithm to reduce the death toll? I can imagine the public demanding it, and the software engineers to start writing code that deals with the rare case where an accident is inevitable. It could start very simple (avoid large groups of people), but could get more advanced over time.
trophycase 8y ago
Speaking un-ethically, the person in the car likely has fewer rights due to agreeing to some or another Term of Service, whereas the people on the street likely have not agreed to such. It may be better to kill the person in the car.
tempest-tost 8y ago
There is no such thing currently as reducing the risk of killing yourself or someone else with your car to 0. Braking distance and speed are insufficient variables to alter given people's preferences about the speed at which they'd wish to arrive at their destination. This will very likely also be the case when autonomous vehicles are deployed. There will be collisions, this is simply unfortunately a fact. Given that there will be collisions, it seems prudent to have the car try to figure out what's better to collide with- the tree or the child. Given that there will be collisions, it also seems prudent to think carefully about other, more difficult moral dilemmas. Refusing to code for an eventuality is itself a moral decision; inaction is an action that has an impact on the world.
- b_tterc_p 8y ago
  There is no probability of impending crash for which it is moral to give up trying to not crash and favor choosing who to kill. Probability is never 1.
RcouF1uZ4gsC 8y ago
I agree with you that this example is so theoretical as to be useless. In addition, focusing on this masks some other more relevant ethical/legal questions. 1. If the car sees a police car flash it's lights for it to pull over, should it pull over unconditionally - not allowing the driver to over-ride? 2. If there is an Amber alert broadcast specifying the car, should it automatically try to alert the authorities as to it's location and/or flag down the nearest police vehicle? 3. Should a car be able to be programmed to automatically avoid going to places that would violate a restriction order on its owner or occupant. 4. If owner/occupant experiences an emergency (heart attack, impending child birth, etc), should the car have a button that will allow it to break traffic laws to try to get to the hospital faster. For example, at 2 AM on a deserted street, do you really need to wait the full time the light is red if you can see that there are no other vehicles?
- maxerickson 8y ago
  Vehicles with telematics are anyway broadcasting their location. Pretty much all recent vehicles have telematics. Most restriction orders aren't worded in a way that could easily be encoded beforehand; they could include language restricting autonomous vehicles... For the emergency situation, if there is a button, it could just interact with the traffic control systems. If there's no traffic at the light, the light could let the car through.
- jimbo1qaz 8y ago
  I would not buy a car that acts against my own interests, or allows the government to deprive me of freedom (the first 3). I don't have a meaningful opinion on #4.
- tsukikage 8y ago
  We have the capability right now to make cars that enforce speed limits on their drivers. We do, in fact, make cars with audible / visible / tactile alerts when the driver exceeds the speed limit for the road they are on, but we seem to have no appetite for buying cars that take the final step to actually automatically brake, never mind requiring all cars to do this. I take this as a very strong indication that overriding driver preferences as in 1-3 will not happen any time soon.
jorjordandan 8y ago
Is this whole "question" really just a mental exercise, like the trolley problem? Surely the more salient question is how much senseless death could be prevented by the adoption of self driving technology, isn't it? This 'conundrum' almost seems to be intended to undercut trust in what could surely be a massive improvement to vehicle safety.
- oliveshell 8y ago
  > Surely the more salient question is how much senseless death could be prevented by the adoption of self driving technology, isn't it? Absolutely. However, when answering that question, I don’t think it’s unreasonable to thoughtfully consider this stuff. Clearly, even flawed autonomous driving— if widely adopted— would save many lives. But I can’t think of many things that would slow that adoption more than a public perception of “killer robots” roaming the streets. If we seriously want wide adoption, it’ll be hard to avoid addressing the qualitative perceptual difference between a death caused by a human driver and one caused by a machine that we’ve engineered.
cortic 8y ago
Should a car, generally prioritize the survival of its occupant or pedestrian? Not answering this is the same as answering it with 'surprise me'. And either answer is wrong.
- rootusrootus 8y ago
  Neither. It should just try to stop. Calculating fatality probabilities is way out of scope.
  - apocalypstyx 8y ago
    But this is just making the choice and then pretending it hasn't been made.
    - deleted 8y ago
      [deleted]
    - rootusrootus 8y ago
      There is no choice to be made, that is the problem with this line of thinking.
      - apocalypstyx 8y ago
        (In this simple case) The vehicle currently traveling toward a stationary object. It has two possible courses of action (sensical action, perhaps would be better, as it could just continue at full speed, but I'll ignore that), but it can either A) attempt to stop before colliding with the object, or B) it can swerve around the object. Obviously, as stated, two possibilities are that if A the driver has a potential for an increased chance of death or injury and if B a third party has a possible increased chance of death or injury. Now, by shunting to A, that makes a statement of priorities: of all the unknowns, the unknowns about the driver (and passengers) and the states that what might result from A are acceptable to the alternative of B. If this were not so, then we could just as easily say that B is equal and acceptable, in that we can no more guarantee the diminshment of an increase in the potential for injury or death for one or the other party(ies) in either event, therefore, both scenarios are equal, unless they're not. And that most people are unwilling to weight B as as acceptable as A, and, say, flip a coin (or its electronic equivalent), it's clear that a judgement call is being made (based upon values) about what is to be prioritized in the situation (or equivalent situations). [The problem is people then take this specific instance of trolley-like problems and dismiss them as abstract construction, and then dismiss and equivalent problems, regardless of real-world applicability.] And if the purpose of autonomous vehicles, as is often stated, is to reduce the numbers of deaths (40,100 in 2017 in the US) and injuries related to such scenarios and vehicles, we have to have at least a tacit understanding of that which we are implicitly prioritizing, if nothing else. However, if the point is just to apply an ideology in the form of a technology, until everything in our lives is computerized, and considered in terms of computation, or if (as I've talked about before on here) it's about removing the need for God and Man to make the choice, then avoiding such questions is the very (inherently not so) explicit goal, as that we can neither admit to either wanting to create the black box nor that it is a black box. The only way to truly not make the decision is to actually not make it, that is, for the motor vehicle to never be traveling in the first place.

mr_toad 8y ago

Agreed. Swerving onto sidewalks or deliberately hitting pedestrians or vehicles is illegal anyway. People talking about self driving ethics should spend less time reading philosophy and more time reading the road rules.

gowld 8y ago

And even if a car was implausibl in the same situation with the same options, it still of course could choose a response randomly or arbitrarily based on how the sorting algorithm happened to rank equal outcomes.

pikma 8y ago

I agree that it's mostly a theoretical problem. But in the long term, if I imagine a world where we all cars are self driving, there would still be accidents from time to time, for things that can't be predicted: a rock falls on the road, an earthquake happens and destroys the road, etc. This would be rare, but would still kill a few thousand people every year. Wouldn't it be natural then to tweak the algorithm to reduce the death toll? I can imagine the public demanding it, and the software engineers to start writing code that deals with the rare case where an accident is inevitable. It could start very simple (avoid large groups of people), but could get more advanced over time.

trophycase 8y ago

Speaking un-ethically, the person in the car likely has fewer rights due to agreeing to some or another Term of Service, whereas the people on the street likely have not agreed to such. It may be better to kill the person in the car.

tempest-tost 8y ago

There is no such thing currently as reducing the risk of killing yourself or someone else with your car to 0. Braking distance and speed are insufficient variables to alter given people's preferences about the speed at which they'd wish to arrive at their destination. This will very likely also be the case when autonomous vehicles are deployed. There will be collisions, this is simply unfortunately a fact. Given that there will be collisions, it seems prudent to have the car try to figure out what's better to collide with- the tree or the child. Given that there will be collisions, it also seems prudent to think carefully about other, more difficult moral dilemmas. Refusing to code for an eventuality is itself a moral decision; inaction is an action that has an impact on the world.

b_tterc_p 8y ago
There is no probability of impending crash for which it is moral to give up trying to not crash and favor choosing who to kill. Probability is never 1.

RcouF1uZ4gsC 8y ago

I agree with you that this example is so theoretical as to be useless. In addition, focusing on this masks some other more relevant ethical/legal questions. 1. If the car sees a police car flash it's lights for it to pull over, should it pull over unconditionally - not allowing the driver to over-ride? 2. If there is an Amber alert broadcast specifying the car, should it automatically try to alert the authorities as to it's location and/or flag down the nearest police vehicle? 3. Should a car be able to be programmed to automatically avoid going to places that would violate a restriction order on its owner or occupant. 4. If owner/occupant experiences an emergency (heart attack, impending child birth, etc), should the car have a button that will allow it to break traffic laws to try to get to the hospital faster. For example, at 2 AM on a deserted street, do you really need to wait the full time the light is red if you can see that there are no other vehicles?

maxerickson 8y ago
Vehicles with telematics are anyway broadcasting their location. Pretty much all recent vehicles have telematics. Most restriction orders aren't worded in a way that could easily be encoded beforehand; they could include language restricting autonomous vehicles... For the emergency situation, if there is a button, it could just interact with the traffic control systems. If there's no traffic at the light, the light could let the car through.
jimbo1qaz 8y ago
I would not buy a car that acts against my own interests, or allows the government to deprive me of freedom (the first 3). I don't have a meaningful opinion on #4.
tsukikage 8y ago
We have the capability right now to make cars that enforce speed limits on their drivers. We do, in fact, make cars with audible / visible / tactile alerts when the driver exceeds the speed limit for the road they are on, but we seem to have no appetite for buying cars that take the final step to actually automatically brake, never mind requiring all cars to do this. I take this as a very strong indication that overriding driver preferences as in 1-3 will not happen any time soon.

jorjordandan 8y ago

Is this whole "question" really just a mental exercise, like the trolley problem? Surely the more salient question is how much senseless death could be prevented by the adoption of self driving technology, isn't it? This 'conundrum' almost seems to be intended to undercut trust in what could surely be a massive improvement to vehicle safety.

oliveshell 8y ago
> Surely the more salient question is how much senseless death could be prevented by the adoption of self driving technology, isn't it? Absolutely. However, when answering that question, I don’t think it’s unreasonable to thoughtfully consider this stuff. Clearly, even flawed autonomous driving— if widely adopted— would save many lives. But I can’t think of many things that would slow that adoption more than a public perception of “killer robots” roaming the streets. If we seriously want wide adoption, it’ll be hard to avoid addressing the qualitative perceptual difference between a death caused by a human driver and one caused by a machine that we’ve engineered.

cortic 8y ago

Should a car, generally prioritize the survival of its occupant or pedestrian? Not answering this is the same as answering it with 'surprise me'. And either answer is wrong.

rootusrootus 8y ago
Neither. It should just try to stop. Calculating fatality probabilities is way out of scope.
- apocalypstyx 8y ago
  But this is just making the choice and then pretending it hasn't been made.
  - deleted 8y ago
    [deleted]
  - rootusrootus 8y ago
    There is no choice to be made, that is the problem with this line of thinking.
    - apocalypstyx 8y ago
      (In this simple case) The vehicle currently traveling toward a stationary object. It has two possible courses of action (sensical action, perhaps would be better, as it could just continue at full speed, but I'll ignore that), but it can either A) attempt to stop before colliding with the object, or B) it can swerve around the object. Obviously, as stated, two possibilities are that if A the driver has a potential for an increased chance of death or injury and if B a third party has a possible increased chance of death or injury. Now, by shunting to A, that makes a statement of priorities: of all the unknowns, the unknowns about the driver (and passengers) and the states that what might result from A are acceptable to the alternative of B. If this were not so, then we could just as easily say that B is equal and acceptable, in that we can no more guarantee the diminshment of an increase in the potential for injury or death for one or the other party(ies) in either event, therefore, both scenarios are equal, unless they're not. And that most people are unwilling to weight B as as acceptable as A, and, say, flip a coin (or its electronic equivalent), it's clear that a judgement call is being made (based upon values) about what is to be prioritized in the situation (or equivalent situations). [The problem is people then take this specific instance of trolley-like problems and dismiss them as abstract construction, and then dismiss and equivalent problems, regardless of real-world applicability.] And if the purpose of autonomous vehicles, as is often stated, is to reduce the numbers of deaths (40,100 in 2017 in the US) and injuries related to such scenarios and vehicles, we have to have at least a tacit understanding of that which we are implicitly prioritizing, if nothing else. However, if the point is just to apply an ideology in the form of a technology, until everything in our lives is computerized, and considered in terms of computation, or if (as I've talked about before on here) it's about removing the need for God and Man to make the choice, then avoiding such questions is the very (inherently not so) explicit goal, as that we can neither admit to either wanting to create the black box nor that it is a black box. The only way to truly not make the decision is to actually not make it, that is, for the motor vehicle to never be traveling in the first place.