5 ms·

Imagine an autonomous car, driving at 60+ mph at a two-lane road (1) which is blocked on both sides and (2) has a pedestrian crossing. (Not sure about 60 mph,

by yongjik 10y ago

Imagine an autonomous car, driving at 60+ mph at a two-lane road (1) which is blocked on both sides and (2) has a pedestrian crossing. (Not sure about 60 mph, but I guess you need to be that fast to reliably (hah!) kill passengers on impact.) We can assume its camera is broken, because it failed to reduce speed (or at least blare the horn) upon seeing a giant concrete block in its path. (Okay, maybe the concrete block fell from the sky when a crane operator failed to secure the load, so the car might have had no time.) And of course the brake is broken. Miraculously, the steering wheel is working, but it's out of question to skid on the side blocks for some reason. Maybe it's actually precipice on either side. (Imagine that: a 60+mph two-lane road, precipices on both sides, with a pedestrian crossing appearing out of nowhere.) Oh, by the way, within 0.5 seconds of seeing the people (remember: the car couldn't see these people until the last moment, otherwise it would have done something!), the car has instant access to their age, sex, profession, and criminal history. The car is made by Google, after all. (Sorry, bad joke.) Q: What is the minimum number of engineering fuckups that should happen to realize this scenario? This is to morality what confiscating baby formula at airport is to national security.

Waterluvian 10y ago
It's a thought experiment.
- yongjik 10y ago
  Thought experiments are supposed to tell us something interesting, by simplifying details while preserving the crux of the matter. Otherwise their values are questionable. I could have asked "If I could dip my head into a black hole and take it out again, what will I see?" That is also a thought experiment, just not a useful one.
  - sesqu 10y ago
    http://www.kurzweilai.net/beyond-interstellar-new-visualizations-of-the-event-horizon-of-black-holes http://www.kurzweilai.net/beyond-interstellar-new-visualizat...
- throwanem 10y ago
  Let us consider a spherical cow...
  - DiabloD3 10y ago
    Oh, right, the kind I can make meatballs out of? (One of my high school teachers would be proud to know that that is possibly the only thing I remember from his class)
- DannyBee 10y ago
  But not, IMHO, a useful one, because it assumes the problems we see today will be the actual real problems in practice. If that's true, i'd go so far to venture a guess that these vehicles will have completely failed. Do people truly believe no one is working on solving the problem of better emergency braking, crash foams, etc? The reason they aren't used is mainly because expense vs value is not there (for manufacturers), not because they don't exist. For example, metal foams (http://onlinepubs.trb.org/onlinepubs/archive/studies/idea/finalreports/highspeedrail/hsr-34final_report.pdf http://onlinepubs.trb.org/onlinepubs/archive/studies/idea/fi... and friends), etc. or hell, making barriers that aren't just concrete! The truth is, if the "moral cost" is high enough, we'll just solve the problem of people dying when they crash in X% of cases, until people/companies feel good about X vs what they pay for X.
  - asimuvPR 10y ago
    I was careful to challenge the usefulness of the exercise. Mostly because I took it as a personal challenge. However, you make an important point. This exercise assumes the problem set will be identical in the future. This sounds like a sensible approach that ends up undermining all of the technological improvements a self driving vehicle will posses. Things like redundant control systems, [V2V, V2I, V2C] networking, run flat tires, and others. The self driving car will be closer to an airplane with a robotic agent as the traffic controller than anything else.
    - nyxtom 10y ago
      Precisely, if it's a problem of trying to get the vehicle to stop, perhaps focus on areas that can increase that probability first (i.e. running flat tires would be a great one to immediately get the car to slow down).
andrei_says_ 10y ago
I couldn't "play" the game for these very reasons. There's no way the player could have all this information. It's a game of "would you rather" pretending to be about self-driving cars. The second question which required a choice between killing homeless people and wealthier ones made me too disgusted to continue.
- esfandia 10y ago
  The system isn't making any value judgment; it's gauging whether /you/ are. And then it compares your value judgments with others. It's interesting in that it has the potential to show your biases, at least relative to the average.
  - daveguy 10y ago
    But the system gauges my value judgement poorly. I never made judgements about whether swerving was a factor, but it gave a gauge on that. I considered the passengers living with witnessing smashed bodies on the windshield (all else equal). Also, even with no difference (or more death ahead), braking and scraping into the wall "left side from observer" drops more momentum safely and allows crossers more time to move forward along their momentum (although I didn't specifically use that because it was against the rules). There is no way to make an optimal decision in this game.
  - Bartweiss 10y ago
    It doesn't have that potential though. There were 13 questions, with each question changing multiple variables. Simply changing the road/legality situation and the demographics between questions buries any informative results. I made my decisions based on a strict decision tree. It covered every case without ever involving demographics (except "cat vs. human"). At the end, I got a halfway accurate summary of my swerving and legality opinions, with a wildly inaccurate but equally confident summary of my demographic opinions. With 13 questions and 5-10 variables, the system couldn't possibly distinguish my ruleset from a totally unrelated one. There aren't enough bits of information to gauge that, and therefore there isn't enough information to gauge much of anything.
fela 10y ago
I don't think this is about creating realistic scenarios, but about finding out what people take into consideration when making moral judgements. The experiment seems to be designed to gather as many such preferences as possible. The hope must be that if people consistently prefer saving the life of young people in this made up scenario they will have similar preferences in a more realistic scenario. Of course weather such a generalization holds will have to be confirmed by further studies. But this seems like a good first step to explore moral decisions more.
- Frompo 10y ago
  This study will learn things about how a biased sample of the population answer choices in a simple game depending on the context given; extrapolation from that basis to anything wider, for example the notion that the players view these choices as "moral" requires far more work. It is well known that people use games for escapism, so it does not seem straightforward that the decisions they make in a game always map cleanly to their real opinions just because you put "moral" in the title of the game. Its also worth keeping in mind that moral decisions have been explored for quite some time, and the novelty here is mainly the mode with which the population is sampled.
LoSboccacc 10y ago
the concrete block might just be a truck engaging the intersection while distracted. sure most moral dilemma of this kind should be resolved by 'install longer range sensor', but other people mistakes are gonna be an important factor in these scenario until all cars are driverless.
deleted 10y ago
[deleted]
deleted 10y ago
[deleted]

by yongjik 10y ago

Waterluvian 10y ago

It's a thought experiment.

yongjik 10y ago
Thought experiments are supposed to tell us something interesting, by simplifying details while preserving the crux of the matter. Otherwise their values are questionable. I could have asked "If I could dip my head into a black hole and take it out again, what will I see?" That is also a thought experiment, just not a useful one.
- sesqu 10y ago
  http://www.kurzweilai.net/beyond-interstellar-new-visualizations-of-the-event-horizon-of-black-holes http://www.kurzweilai.net/beyond-interstellar-new-visualizat...
throwanem 10y ago
Let us consider a spherical cow...
- DiabloD3 10y ago
  Oh, right, the kind I can make meatballs out of? (One of my high school teachers would be proud to know that that is possibly the only thing I remember from his class)
DannyBee 10y ago
But not, IMHO, a useful one, because it assumes the problems we see today will be the actual real problems in practice. If that's true, i'd go so far to venture a guess that these vehicles will have completely failed. Do people truly believe no one is working on solving the problem of better emergency braking, crash foams, etc? The reason they aren't used is mainly because expense vs value is not there (for manufacturers), not because they don't exist. For example, metal foams (http://onlinepubs.trb.org/onlinepubs/archive/studies/idea/finalreports/highspeedrail/hsr-34final_report.pdf http://onlinepubs.trb.org/onlinepubs/archive/studies/idea/fi... and friends), etc. or hell, making barriers that aren't just concrete! The truth is, if the "moral cost" is high enough, we'll just solve the problem of people dying when they crash in X% of cases, until people/companies feel good about X vs what they pay for X.
- asimuvPR 10y ago
  I was careful to challenge the usefulness of the exercise. Mostly because I took it as a personal challenge. However, you make an important point. This exercise assumes the problem set will be identical in the future. This sounds like a sensible approach that ends up undermining all of the technological improvements a self driving vehicle will posses. Things like redundant control systems, [V2V, V2I, V2C] networking, run flat tires, and others. The self driving car will be closer to an airplane with a robotic agent as the traffic controller than anything else.
  - nyxtom 10y ago
    Precisely, if it's a problem of trying to get the vehicle to stop, perhaps focus on areas that can increase that probability first (i.e. running flat tires would be a great one to immediately get the car to slow down).

andrei_says_ 10y ago

I couldn't "play" the game for these very reasons. There's no way the player could have all this information. It's a game of "would you rather" pretending to be about self-driving cars. The second question which required a choice between killing homeless people and wealthier ones made me too disgusted to continue.

esfandia 10y ago
The system isn't making any value judgment; it's gauging whether /you/ are. And then it compares your value judgments with others. It's interesting in that it has the potential to show your biases, at least relative to the average.
- daveguy 10y ago
  But the system gauges my value judgement poorly. I never made judgements about whether swerving was a factor, but it gave a gauge on that. I considered the passengers living with witnessing smashed bodies on the windshield (all else equal). Also, even with no difference (or more death ahead), braking and scraping into the wall "left side from observer" drops more momentum safely and allows crossers more time to move forward along their momentum (although I didn't specifically use that because it was against the rules). There is no way to make an optimal decision in this game.
- Bartweiss 10y ago
  It doesn't have that potential though. There were 13 questions, with each question changing multiple variables. Simply changing the road/legality situation and the demographics between questions buries any informative results. I made my decisions based on a strict decision tree. It covered every case without ever involving demographics (except "cat vs. human"). At the end, I got a halfway accurate summary of my swerving and legality opinions, with a wildly inaccurate but equally confident summary of my demographic opinions. With 13 questions and 5-10 variables, the system couldn't possibly distinguish my ruleset from a totally unrelated one. There aren't enough bits of information to gauge that, and therefore there isn't enough information to gauge much of anything.

fela 10y ago

I don't think this is about creating realistic scenarios, but about finding out what people take into consideration when making moral judgements. The experiment seems to be designed to gather as many such preferences as possible. The hope must be that if people consistently prefer saving the life of young people in this made up scenario they will have similar preferences in a more realistic scenario. Of course weather such a generalization holds will have to be confirmed by further studies. But this seems like a good first step to explore moral decisions more.

Frompo 10y ago
This study will learn things about how a biased sample of the population answer choices in a simple game depending on the context given; extrapolation from that basis to anything wider, for example the notion that the players view these choices as "moral" requires far more work. It is well known that people use games for escapism, so it does not seem straightforward that the decisions they make in a game always map cleanly to their real opinions just because you put "moral" in the title of the game. Its also worth keeping in mind that moral decisions have been explored for quite some time, and the novelty here is mainly the mode with which the population is sampled.

LoSboccacc 10y ago

the concrete block might just be a truck engaging the intersection while distracted. sure most moral dilemma of this kind should be resolved by 'install longer range sensor', but other people mistakes are gonna be an important factor in these scenario until all cars are driverless.

deleted 10y ago

[deleted]

deleted 10y ago

[deleted]