8 ms·

A Study on Driverless-Car Ethics

39 points by theBashShell 8y ago

minikites 8y ago
https://idlewords.com/talks/sase_panel.htm https://idlewords.com/talks/sase_panel.htm >Machine learning is like money laundering for bias. It's a clean, mathematical apparatus that gives the status quo the aura of logical inevitability.
- gowld 8y ago
  It used to be that Ivy League colleges laundered privelege into elite degrees for people who would become consultants who gave an aura of legitimacy to biased decisions. Maybe the robots will put them out of a job.
  - b_tterc_p 8y ago
    Consultants are the ones legitimizing the robots
vishvananda 8y ago
Is it just me or does solving the moral dilemma of prioritization based on attributes of the person seem like a worthless endeavor? In practice, I can't come up with a case where it would matter. For example, a more reasonable metric for a machine to use is the probability of injury/death. If swerving is 90% likely to kill a person and staying straight is only 89% likely, then staying straight is a better choice. I don't see how attributes of the person would ever trump the probability of harm. The cases where probability is roughly equal for multiple actions will be incredibly rare.
- jjoonathan 8y ago
  It's the ultimate bikeshed. The only winning move is not to play.
- Spooky23 8y ago
  What’s the reliability of the risk assessment? If you child died because there was a 1% higher projected chance that two people would be killed in a collision... would that seem just? I think the insistence on experimenting with autonomous driving outside of controlled access highways is insane.
  - dnbgfher 8y ago
    And what about the family of those two people? I don't understand the insistence of bringing everything back to "what if it was your child." Pretty much everybody has family, and friends, and all that. I'm not sure how somebody being a child of somebody is supposed to be a rational argument for adjusting any of our thinking. As far as I can tell, it's basically the ultimate appeal to emotion.
    - Spooky23 8y ago
      It’s a call to humanity, not emotion. When you’re talking about selecting who dies with a 1% margin with some ML process with an unknown margin of error, I question your judgement.
      - dnbgfher 8y ago
        I don't see how it's a call to humanity. "Think if it was your kid vs two strangers" isn't calling on humanity. It's calling on familial tribalism and selfishness if anything. And obviously, in the real world, a ML-given estimate with a 1% variance is probably entirely useless. In these sorts of hypotheticals,I'm not sure why that really matters. You can play with the numbers as you want, or even move the whole question out far enough into the future that the margin of error can be considered very low. The question remains the same.
  - maxerickson 8y ago
    We let teenagers drive. I think it is easy enough to predict that all the systems will do is prioritize avoiding pedestrians while minimizing collision energy otherwise. A 30 mph collision will mess up a pedestrian. It won't be a big deal for restrained passengers. If it is frequently the case that autonomous systems have to pick between pedestrians, that points to traffic control changes, not a philosophy engine in the car.
- ordu 8y ago
  89% percent chance to kill homeless person or 90% chance to kill pregnant woman? > I don't see how attributes of the person would ever trump the probability of harm. Maybe they shouldn't, maybe they should. How do we know? What are the terminal values? Society effectiveness? Or equality of all humans? Or minimizing public outcry in cases when autonomous car killed someone? > The cases where probability is roughly equal for multiple actions will be incredibly rare. With millions of autonomous cars on streets rare outcomes would happen all the time.
  - x3n0ph3n3 8y ago
    Do you expect a person to be able to solve that problem? Would you fault a person for not solving it the way you want it solved? If not, why should a driverless car?
  - brokenmachine 8y ago
    >89% percent chance to kill homeless person or 90% chance to kill pregnant woman? Just fat, or pregnant? We need to know! Get onto it, engineers! The car's algorithm for deciding this question is a matter of life or death! /s of course, because I think this whole philosophical argument about self-driving cars is ridiculous when our current tech can't even reliably determine if an object is a stationary barrier or not.
- sokoloff 8y ago
  I will posit that killing a 9-year old or a 29-year old is actually worse than killing a 99-year old, all else being equal. (I would hold this view even if I were the 99-year old.)
b_tterc_p 8y ago
“ if a car detects a sudden obstacle—say, a jackknifed truck—should it hit the truck and kill its own driver, or should it swerve onto a crowded sidewalk and kill pedestrians? A human driver might react randomly (if she has time to react at all), but the response of an autonomous vehicle would have to be programmed ahead of time. ” I hate this logic. No, it didn’t need to be programmed ahead of time. It should try to avoid crashing. It shouldn’t try to solve a philosophical problem. We don’t want to require our cars to observe it’s surroundings and calculate number of things identified as humans to minimize expected loss of life. That’s difficult and error prone. If you’re going down a street and something jumps in your way, make the program try not to hit anything. Ideally by braking, because you shouldn’t be going fast enough to not be able to do so based on your environment.
- mr_toad 8y ago
  Agreed. Swerving onto sidewalks or deliberately hitting pedestrians or vehicles is illegal anyway. People talking about self driving ethics should spend less time reading philosophy and more time reading the road rules.
- gowld 8y ago
  And even if a car was implausibl in the same situation with the same options, it still of course could choose a response randomly or arbitrarily based on how the sorting algorithm happened to rank equal outcomes.
- pikma 8y ago
  I agree that it's mostly a theoretical problem. But in the long term, if I imagine a world where we all cars are self driving, there would still be accidents from time to time, for things that can't be predicted: a rock falls on the road, an earthquake happens and destroys the road, etc. This would be rare, but would still kill a few thousand people every year. Wouldn't it be natural then to tweak the algorithm to reduce the death toll? I can imagine the public demanding it, and the software engineers to start writing code that deals with the rare case where an accident is inevitable. It could start very simple (avoid large groups of people), but could get more advanced over time.
- 8y ago
gowld 8y ago
Does this have anything to do with driverless cars? Driverless cars will not be programmed to target victims by race, and driverless cars' main benefit is reaction time to take simple safe actions like stopping the vehicle or turning into an open space. The car systems won't have elaborate neural nets for deciding when to jump over a pylon onto a train track.
- minikites 8y ago
  >Driverless cars will not be programmed to target victims by race Not on purpose, but "machine learning" will have the same outcome: https://theoutline.com/post/7022/ai-trolley-problem-ethics https://theoutline.com/post/7022/ai-trolley-problem-ethics https://twitter.com/wef/status/1058675216027660288 https://twitter.com/wef/status/1058675216027660288
- wizardforhire 8y ago
  You say this and I want to agree... but then again who are the people building driverless cars? Unless the software is opensource how are we supposed to know? Are we so naive that we’re going to take any company’s word for it? ...and then are we so naive that we’re going to fall for the obvious plant arguments in forums against it? If the internet has taught me anything it’s that you can’t trust anyone.
Pulcinella 8y ago
These discussions always bring up the Trolley Problem which I find to be pretty insidious. The trolley problem is designed to hide the real moral issue. Here is the original Trolley Problem for comparison. “Suppose that a judge or magistrate is faced with rioters demanding that a culprit be found for a certain crime and threatening otherwise to take their own bloody revenge on a particular section of the community. The real culprit being unknown, the judge sees himself as able to prevent the bloodshed only by framing some innocent person and having him executed. Beside this example is placed another in which a pilot whose airplane is about to crash is deciding whether to steer from a more to a less inhabited area. To make the parallel as close as possible it may rather be supposed that he is the driver of a runaway tram which he can only steer from one narrow track on to another; five men are working on one track and one man on the other; anyone on the track he enters is bound to be killed. In the case of the riots the mob have five hostages, so that in both examples the exchange is supposed to be one man's life for the lives of five.” The whole thing starts with a pretty elitist framework, that the best way to run society is to lie to the public because they can’t be trusted. The problem then ends with the infamous trolley problem which is so stripped down of any context it’s basically a dark pattern. You are expected to choose who is to die instead of wondering why do people keep getting run over by trolleys? Why does anyone have to die? Why can’t the track workers have safe working conditions?
hughes 8y ago
The Trolley Problem is the most boring part of the ethics of autonomous driving. Would you tell a human driver to stay off the roads until they decide in advance what they'd do in such a ludicrous situation?
- dingaling 8y ago
  The difference with a human driver is that there is no point asking them to decide ahead of time. In all but the most highly-trained soldiers, what a person will do in the case of immediate fatal personal threat can't be preprogrammed.
hnaccy 8y ago
Can the car communicate over local network and check the pedestrians' credit scores?
jijji 8y ago
ahh a pseudo-science article coming from The New Yorker. A comic strip from Dilbert would give more insightful information.
- taneq 8y ago
  Well, he's recently started talking about self-driving cars, so it's on the cards: https://dilbert.com/strip/2019-01-25 https://dilbert.com/strip/2019-01-25
travisoneill1 8y ago
These problems are always brought as a drawback to driverless cars but are completely irrelevant. How many "trolley" situations are there compared to the thousands of times where the choice is: A. Human error causes cars to collide and kills people B. The cars don't collide
csours 8y ago
I think philosophy and culture are important, but this meme is very silly. Personifying self-driving systems is a great disservice and misleading. A self driving car detecting a sudden obstacle will not have time to classify anything to the extent that these stories have in mind, and certainly not to the point that humans would (leaving aside the fact that humans also would not have time to classify the items at a "sudden" obstacle). --- That being said, there are very real ethical questions in the self driving world: 1. Just because you can commute farther, should you? 2. Should more expensive cars be able to disrupt traffic for individual benefit, or should all self driving systems follow a consistent set of rules of the road to maximize throughput? 2.1 Should I be able to tweak my car to be a little more dangerous, but also quicker? 3. Should carpooling be mandated? Encouraged through tax breaks? subsidies? 4. Should self driving cars be allowed to cut through neighborhoods or be required to follow major roads around them? What about parking lots? I'm sure there are other really interesting questions as well, but the trolley problem is not one of them.
- warent 8y ago
  Well in the case of point 2, when you have a lot of vehicles disrupting traffic for individual benefit, what you end up with is traffic as we have it today--horribly inefficient and slow for everyone. When everyone follows a consistent set of rules to maximize throughput, that's what actually leads to the maximum individual benefit.
- maxerickson 8y ago
  The answers to 2, 2 and 4 are pretty clearly no. For 1 and the first 2, dynamic tolls are the smart solution. Also applies to the second 2. If people want to go faster, let's tax them for it.
gnicholas 8y ago
Ah, the perfect way for Tesla to segment their different product lines — make the expensive ones more “selfish” in their accident mitigation algorithms.
trainingaccount 8y ago
Hope they will be taking minutes in all the meetings where the business requirements are collected.
xiphias2 8y ago
The real ethical question is not asked: should a driverless car be allowed on the street if it waits 10 minutes at the left turn while making a traffic jam? Real autonomous cars are already making real problems. I believe it's Waymo's job to take over the left turns if the waiting line is too long.
masonic 8y ago
A concern of mine is that in a situation where an autonomous vehicle is "at fault", its owner has a strong incentive to have it flee the scene. If "caught", the liability exposure is the same, so there's no downside. You can't incarcerate an algorithm or even take its license.

39 points by theBashShell 8y ago

https://idlewords.com/talks/sase_panel.htm https://idlewords.com/talks/sase_panel.htm >Machine learning is like money laundering for bias. It's a clean, mathematical apparatus that gives the status quo the aura of logical inevitability.

gowld 8y ago
It used to be that Ivy League colleges laundered privelege into elite degrees for people who would become consultants who gave an aura of legitimacy to biased decisions. Maybe the robots will put them out of a job.
- b_tterc_p 8y ago
  Consultants are the ones legitimizing the robots

vishvananda 8y ago

Is it just me or does solving the moral dilemma of prioritization based on attributes of the person seem like a worthless endeavor? In practice, I can't come up with a case where it would matter. For example, a more reasonable metric for a machine to use is the probability of injury/death. If swerving is 90% likely to kill a person and staying straight is only 89% likely, then staying straight is a better choice. I don't see how attributes of the person would ever trump the probability of harm. The cases where probability is roughly equal for multiple actions will be incredibly rare.

jjoonathan 8y ago
It's the ultimate bikeshed. The only winning move is not to play.
Spooky23 8y ago
What’s the reliability of the risk assessment? If you child died because there was a 1% higher projected chance that two people would be killed in a collision... would that seem just? I think the insistence on experimenting with autonomous driving outside of controlled access highways is insane.
- dnbgfher 8y ago
  And what about the family of those two people? I don't understand the insistence of bringing everything back to "what if it was your child." Pretty much everybody has family, and friends, and all that. I'm not sure how somebody being a child of somebody is supposed to be a rational argument for adjusting any of our thinking. As far as I can tell, it's basically the ultimate appeal to emotion.
  - Spooky23 8y ago
    It’s a call to humanity, not emotion. When you’re talking about selecting who dies with a 1% margin with some ML process with an unknown margin of error, I question your judgement.
    - dnbgfher 8y ago
      I don't see how it's a call to humanity. "Think if it was your kid vs two strangers" isn't calling on humanity. It's calling on familial tribalism and selfishness if anything. And obviously, in the real world, a ML-given estimate with a 1% variance is probably entirely useless. In these sorts of hypotheticals,I'm not sure why that really matters. You can play with the numbers as you want, or even move the whole question out far enough into the future that the margin of error can be considered very low. The question remains the same.
- maxerickson 8y ago
  We let teenagers drive. I think it is easy enough to predict that all the systems will do is prioritize avoiding pedestrians while minimizing collision energy otherwise. A 30 mph collision will mess up a pedestrian. It won't be a big deal for restrained passengers. If it is frequently the case that autonomous systems have to pick between pedestrians, that points to traffic control changes, not a philosophy engine in the car.
ordu 8y ago
89% percent chance to kill homeless person or 90% chance to kill pregnant woman? > I don't see how attributes of the person would ever trump the probability of harm. Maybe they shouldn't, maybe they should. How do we know? What are the terminal values? Society effectiveness? Or equality of all humans? Or minimizing public outcry in cases when autonomous car killed someone? > The cases where probability is roughly equal for multiple actions will be incredibly rare. With millions of autonomous cars on streets rare outcomes would happen all the time.
- x3n0ph3n3 8y ago
  Do you expect a person to be able to solve that problem? Would you fault a person for not solving it the way you want it solved? If not, why should a driverless car?
- brokenmachine 8y ago
  >89% percent chance to kill homeless person or 90% chance to kill pregnant woman? Just fat, or pregnant? We need to know! Get onto it, engineers! The car's algorithm for deciding this question is a matter of life or death! /s of course, because I think this whole philosophical argument about self-driving cars is ridiculous when our current tech can't even reliably determine if an object is a stationary barrier or not.
sokoloff 8y ago
I will posit that killing a 9-year old or a 29-year old is actually worse than killing a 99-year old, all else being equal. (I would hold this view even if I were the 99-year old.)

b_tterc_p 8y ago

“ if a car detects a sudden obstacle—say, a jackknifed truck—should it hit the truck and kill its own driver, or should it swerve onto a crowded sidewalk and kill pedestrians? A human driver might react randomly (if she has time to react at all), but the response of an autonomous vehicle would have to be programmed ahead of time. ” I hate this logic. No, it didn’t need to be programmed ahead of time. It should try to avoid crashing. It shouldn’t try to solve a philosophical problem. We don’t want to require our cars to observe it’s surroundings and calculate number of things identified as humans to minimize expected loss of life. That’s difficult and error prone. If you’re going down a street and something jumps in your way, make the program try not to hit anything. Ideally by braking, because you shouldn’t be going fast enough to not be able to do so based on your environment.

mr_toad 8y ago
Agreed. Swerving onto sidewalks or deliberately hitting pedestrians or vehicles is illegal anyway. People talking about self driving ethics should spend less time reading philosophy and more time reading the road rules.
gowld 8y ago
And even if a car was implausibl in the same situation with the same options, it still of course could choose a response randomly or arbitrarily based on how the sorting algorithm happened to rank equal outcomes.
pikma 8y ago
I agree that it's mostly a theoretical problem. But in the long term, if I imagine a world where we all cars are self driving, there would still be accidents from time to time, for things that can't be predicted: a rock falls on the road, an earthquake happens and destroys the road, etc. This would be rare, but would still kill a few thousand people every year. Wouldn't it be natural then to tweak the algorithm to reduce the death toll? I can imagine the public demanding it, and the software engineers to start writing code that deals with the rare case where an accident is inevitable. It could start very simple (avoid large groups of people), but could get more advanced over time.
8y ago

gowld 8y ago

Does this have anything to do with driverless cars? Driverless cars will not be programmed to target victims by race, and driverless cars' main benefit is reaction time to take simple safe actions like stopping the vehicle or turning into an open space. The car systems won't have elaborate neural nets for deciding when to jump over a pylon onto a train track.

minikites 8y ago
>Driverless cars will not be programmed to target victims by race Not on purpose, but "machine learning" will have the same outcome: https://theoutline.com/post/7022/ai-trolley-problem-ethics https://theoutline.com/post/7022/ai-trolley-problem-ethics https://twitter.com/wef/status/1058675216027660288 https://twitter.com/wef/status/1058675216027660288
wizardforhire 8y ago
You say this and I want to agree... but then again who are the people building driverless cars? Unless the software is opensource how are we supposed to know? Are we so naive that we’re going to take any company’s word for it? ...and then are we so naive that we’re going to fall for the obvious plant arguments in forums against it? If the internet has taught me anything it’s that you can’t trust anyone.

Pulcinella 8y ago

These discussions always bring up the Trolley Problem which I find to be pretty insidious. The trolley problem is designed to hide the real moral issue. Here is the original Trolley Problem for comparison. “Suppose that a judge or magistrate is faced with rioters demanding that a culprit be found for a certain crime and threatening otherwise to take their own bloody revenge on a particular section of the community. The real culprit being unknown, the judge sees himself as able to prevent the bloodshed only by framing some innocent person and having him executed. Beside this example is placed another in which a pilot whose airplane is about to crash is deciding whether to steer from a more to a less inhabited area. To make the parallel as close as possible it may rather be supposed that he is the driver of a runaway tram which he can only steer from one narrow track on to another; five men are working on one track and one man on the other; anyone on the track he enters is bound to be killed. In the case of the riots the mob have five hostages, so that in both examples the exchange is supposed to be one man's life for the lives of five.” The whole thing starts with a pretty elitist framework, that the best way to run society is to lie to the public because they can’t be trusted. The problem then ends with the infamous trolley problem which is so stripped down of any context it’s basically a dark pattern. You are expected to choose who is to die instead of wondering why do people keep getting run over by trolleys? Why does anyone have to die? Why can’t the track workers have safe working conditions?

hughes 8y ago

The Trolley Problem is the most boring part of the ethics of autonomous driving. Would you tell a human driver to stay off the roads until they decide in advance what they'd do in such a ludicrous situation?

dingaling 8y ago
The difference with a human driver is that there is no point asking them to decide ahead of time. In all but the most highly-trained soldiers, what a person will do in the case of immediate fatal personal threat can't be preprogrammed.

hnaccy 8y ago

Can the car communicate over local network and check the pedestrians' credit scores?

jijji 8y ago

ahh a pseudo-science article coming from The New Yorker. A comic strip from Dilbert would give more insightful information.

taneq 8y ago
Well, he's recently started talking about self-driving cars, so it's on the cards: https://dilbert.com/strip/2019-01-25 https://dilbert.com/strip/2019-01-25

travisoneill1 8y ago

These problems are always brought as a drawback to driverless cars but are completely irrelevant. How many "trolley" situations are there compared to the thousands of times where the choice is: A. Human error causes cars to collide and kills people B. The cars don't collide

csours 8y ago

I think philosophy and culture are important, but this meme is very silly. Personifying self-driving systems is a great disservice and misleading. A self driving car detecting a sudden obstacle will not have time to classify anything to the extent that these stories have in mind, and certainly not to the point that humans would (leaving aside the fact that humans also would not have time to classify the items at a "sudden" obstacle). --- That being said, there are very real ethical questions in the self driving world: 1. Just because you can commute farther, should you? 2. Should more expensive cars be able to disrupt traffic for individual benefit, or should all self driving systems follow a consistent set of rules of the road to maximize throughput? 2.1 Should I be able to tweak my car to be a little more dangerous, but also quicker? 3. Should carpooling be mandated? Encouraged through tax breaks? subsidies? 4. Should self driving cars be allowed to cut through neighborhoods or be required to follow major roads around them? What about parking lots? I'm sure there are other really interesting questions as well, but the trolley problem is not one of them.

warent 8y ago
Well in the case of point 2, when you have a lot of vehicles disrupting traffic for individual benefit, what you end up with is traffic as we have it today--horribly inefficient and slow for everyone. When everyone follows a consistent set of rules to maximize throughput, that's what actually leads to the maximum individual benefit.
maxerickson 8y ago
The answers to 2, 2 and 4 are pretty clearly no. For 1 and the first 2, dynamic tolls are the smart solution. Also applies to the second 2. If people want to go faster, let's tax them for it.

gnicholas 8y ago

Ah, the perfect way for Tesla to segment their different product lines — make the expensive ones more “selfish” in their accident mitigation algorithms.

trainingaccount 8y ago

Hope they will be taking minutes in all the meetings where the business requirements are collected.

xiphias2 8y ago

The real ethical question is not asked: should a driverless car be allowed on the street if it waits 10 minutes at the left turn while making a traffic jam? Real autonomous cars are already making real problems. I believe it's Waymo's job to take over the left turns if the waiting line is too long.

masonic 8y ago

A concern of mine is that in a situation where an autonomous vehicle is "at fault", its owner has a strong incentive to have it flee the scene. If "caught", the liability exposure is the same, so there's no downside. You can't incarcerate an algorithm or even take its license.