4 ms·

The trolley problem is a tough one.

by kxrm 8y ago

x3n0ph3n3 8y ago
It has nothing to do with the trolley problem. The trolley problem is a headline-grabbing non-problem in the actual design of driverless cars.
- imtringued 8y ago
  The real trolley problem: [ ] Let self driving cars kill people during development but try to make it up by saving thousands of lives after they are perfected. (Uber) [ ] Make self driving cars extremely safe from the start but more people end up dying from manual driving because development takes longer. (Waymo)
  - Piskvorrr 8y ago
    The underlying assumption "general autonomous driving can be made much safer in a short timeframe [years]" is not granted. It may be that the first option could have a death toll higher than what it would prevent - but the thing is, we don't know, it's an unknown, an assumption "it will be much better once everybody drinks the koolaid."
spin 8y ago
In my entire life, I've never encountered the Trolley Problem, and I've never met anyone who has. Most safety problems in driving can be solved by slowing down. The problem with AI is all those weird, little edge cases that humans can reason through -- for example: if there's a deer next to the road, then I'll slow down, even if it's not on the road yet. I've known many people who have hit a deer when it spontaneously jumps into traffic. Or something like: someone's not quite staying in their own lane, so I have to be careful when I pass them.
- batteryhorse 8y ago
  Perhaps most safety problems. But if you need to outrun an erupting volcano, a tsunami, or the police, slowing down won't really help you.
  - spin 8y ago
    "... outrun ... the police" I doubt that this is a use-case that legal, for-profit companies will be pursuing. For the other use-cases, you can just say: Manual driving only.
  - spin 8y ago
    I think that this is a more realistic/relevant example of "slowing down or stopping is not always the safest action": https://www.youtube.com/watch?v=eWuK-fi-D_w https://www.youtube.com/watch?v=eWuK-fi-D_w
rootusrootus 8y ago
I've never understood why. I can't imagine any programmer or self-driving car company having any interest whatsoever in going down that particular rabbit hole. The answer is simple -- we don't expect humans to do realtime analysis of who is worth killing when an accident is happening, so we will not expect that of computers either. It's an unsolvable problem. So we just tell the computer that in the event that an accident appears unavoidable it should do everything within it's power to stop the car immediately. End of story.
- skgoa 8y ago
  And this kind of evaluation of one life against another is completely illegal in Germany. I.e. the german automotive industry legally can not make such systems in their automated cars and no cars operating in Germany can ave these systems either. I fully exect other countries to enact similar laws, if it ever becomes necessary.
- Piskvorrr 8y ago
  It's a thought experiment; do not expect a literal trolley (or even literal corpses!) any more than Schrödinger's cat deals with actual felines. Here's a restatement without all those corpses: In the event that an accident appears unavoidable, the computer should do everything within it's power to stop the car immediately. (This is already a given: safety first.) "If there are multiple ways of doing the above, what variables should the computer optimize for - stopping distance alone, or stopping so that it doesn't immediately get rammed from behind?" There's your trolley problem again, just restated so it doesn't appear so offensive: in both cases, the occupants of the vehicle are in danger, as are the occupants of nearby vehicles. Now is the interest in the rabbit-hole clearer? The problem doesn't go away just because it's inconvenient to solve...
  - rootusrootus 8y ago
    Well, that's where crumple zones come into play. Trying to have a computer calculate whether or not it is feasible to stop slower to avoid or reduce an imminent rear collision gets very complex in a hurry. What if the guy behind you is much smaller, or much bigger, what if his car has better crumple zones, or none, what kind of energy is he going to impart to you when he hits and where will that send you, who is at that location, etc. These are the kind of questions that can't be answered with certainty even when you give a supercomputer a few days to work it all out. So I think the only reasonable solution is to not expect self-driving cars to make any kind of moral decision. As a bonus it is much easier to explain in court and saves us endless pontificating on whether or not the computer made the right choice and who is actually responsible if it did not. We tell humans not to swerve for squirrels, because it's a great way to end up dead, so the computer should get the same instruction. Slow down as you can, the prime directive is to maintain control.

The trolley problem is a tough one.

by kxrm 8y ago

The trolley problem is a tough one.

It has nothing to do with the trolley problem. The trolley problem is a headline-grabbing non-problem in the actual design of driverless cars.

imtringued 8y ago
The real trolley problem: [ ] Let self driving cars kill people during development but try to make it up by saving thousands of lives after they are perfected. (Uber) [ ] Make self driving cars extremely safe from the start but more people end up dying from manual driving because development takes longer. (Waymo)
- Piskvorrr 8y ago
  The underlying assumption "general autonomous driving can be made much safer in a short timeframe [years]" is not granted. It may be that the first option could have a death toll higher than what it would prevent - but the thing is, we don't know, it's an unknown, an assumption "it will be much better once everybody drinks the koolaid."

spin 8y ago

In my entire life, I've never encountered the Trolley Problem, and I've never met anyone who has. Most safety problems in driving can be solved by slowing down. The problem with AI is all those weird, little edge cases that humans can reason through -- for example: if there's a deer next to the road, then I'll slow down, even if it's not on the road yet. I've known many people who have hit a deer when it spontaneously jumps into traffic. Or something like: someone's not quite staying in their own lane, so I have to be careful when I pass them.

batteryhorse 8y ago
Perhaps most safety problems. But if you need to outrun an erupting volcano, a tsunami, or the police, slowing down won't really help you.
- spin 8y ago
  "... outrun ... the police" I doubt that this is a use-case that legal, for-profit companies will be pursuing. For the other use-cases, you can just say: Manual driving only.
- spin 8y ago
  I think that this is a more realistic/relevant example of "slowing down or stopping is not always the safest action": https://www.youtube.com/watch?v=eWuK-fi-D_w https://www.youtube.com/watch?v=eWuK-fi-D_w

rootusrootus 8y ago

I've never understood why. I can't imagine any programmer or self-driving car company having any interest whatsoever in going down that particular rabbit hole. The answer is simple -- we don't expect humans to do realtime analysis of who is worth killing when an accident is happening, so we will not expect that of computers either. It's an unsolvable problem. So we just tell the computer that in the event that an accident appears unavoidable it should do everything within it's power to stop the car immediately. End of story.

skgoa 8y ago
And this kind of evaluation of one life against another is completely illegal in Germany. I.e. the german automotive industry legally can not make such systems in their automated cars and no cars operating in Germany can ave these systems either. I fully exect other countries to enact similar laws, if it ever becomes necessary.
Piskvorrr 8y ago
It's a thought experiment; do not expect a literal trolley (or even literal corpses!) any more than Schrödinger's cat deals with actual felines. Here's a restatement without all those corpses: In the event that an accident appears unavoidable, the computer should do everything within it's power to stop the car immediately. (This is already a given: safety first.) "If there are multiple ways of doing the above, what variables should the computer optimize for - stopping distance alone, or stopping so that it doesn't immediately get rammed from behind?" There's your trolley problem again, just restated so it doesn't appear so offensive: in both cases, the occupants of the vehicle are in danger, as are the occupants of nearby vehicles. Now is the interest in the rabbit-hole clearer? The problem doesn't go away just because it's inconvenient to solve...
- rootusrootus 8y ago
  Well, that's where crumple zones come into play. Trying to have a computer calculate whether or not it is feasible to stop slower to avoid or reduce an imminent rear collision gets very complex in a hurry. What if the guy behind you is much smaller, or much bigger, what if his car has better crumple zones, or none, what kind of energy is he going to impart to you when he hits and where will that send you, who is at that location, etc. These are the kind of questions that can't be answered with certainty even when you give a supercomputer a few days to work it all out. So I think the only reasonable solution is to not expect self-driving cars to make any kind of moral decision. As a bonus it is much easier to explain in court and saves us endless pontificating on whether or not the computer made the right choice and who is actually responsible if it did not. We tell humans not to swerve for squirrels, because it's a great way to end up dead, so the computer should get the same instruction. Slow down as you can, the prime directive is to maintain control.