6 ms·

More likely than one might want to think. A car traveling the same direction swerves into your lane, crowding you off the road. Does your self-driving car all

More likely than one might want to think. A car traveling the same direction swerves into your lane, crowding you off the road. Does your self-driving car allow the collision and push the offending car back in order to hold your lane? Or does it avoid the collision and run you off the road? (As a person who drives in the mountains, this could have deadly results. If you live in the plains of Kansas, maybe not so much.)

marcosdumay 10y ago
It does not run off the road... And it pushes the break hard. What kind of ethical dilemma should it have on that example?
- msandford 10y ago
  It has 0.5 seconds until collision so no hope of stopping. Push the brake all you want, unless that car gets back in its lane, people die.
  - rickycook 10y ago
    .5 seconds is both incorrect, and plenty for a car that sees and responds instantly to make a car-length braking correction.
    - WalterBright 10y ago
      A collision may not be avoidable, but every decrement to the speed will reduce its severity. It's the same idea as having crumple zones on a car body.
  - Retra 10y ago
    Do you have a specific example of this situation occurring as you've described it? If so, was there an ethical dilemma when a human was in control? If not, don't worry about it.
saidajigumi 10y ago
Yes, this situation is in the realm of the plausible, which has little to do with my comment. I'm not arguing against an analysis of the decisions made by a self-driving system. That's essential. I'm arguing against applying ethical models that utilize knowledge outside the realm of the knowledge or "reasoning" capacity of the drive algorithms. To continue your example, I'll pose that any system with unilateral behavior of self-endangerment is extremely problematic. Motivating example: assume that self-driving cars will always avoid head-on collisions with other vehicles, even if there's no safe alternative. Including driving off a cliff. Attack: manual driver drives down the wrong side of the road (or maybe the middle of a two-lane road), forcing all of the self-driving cars off the cliff during evening commute hours to a hillside residential neighborhood. I suspect all systems that don't hew to some baseline of self-preservation are subject to similar modes of attack.
- schoen 10y ago
  There's a whole kind of unpleasant but interesting topic about attacks against behavior of self-driving cars. I think I saw an earlier HN thread that suggested that they might put occupants at greater risk of kidnapping because the cars could refuse to take evasive actions that endanger or injure a kidnapper (or just violate a traffic law). This risk might be farfetched, but it may illustrate the concern about people knowing the limits that self-driving cars will respect and then exploiting those limits in some way.
  - schoen 10y ago
    > I think I saw an earlier HN thread that suggested I might also have heard about some of these problems from Brad Templeton, who describes "Abusing robocar collision-avoidance" among many other interesting problems at http://www.templetons.com/brad/robocars/downsides.html http://www.templetons.com/brad/robocars/downsides.html I don't remember if he also mentioned the kidnapping scenario when I heard him present about this.
  - imtringued 10y ago
    >I think I saw an earlier HN thread that suggested that they might put occupants at greater risk of kidnapping because the cars could refuse to take evasive actions that endanger or injure a kidnapper What alternative would a human take? Would they kill the kidnappers by driving over them?
datenwolf 10y ago
> Does your self-driving car allow the collision and push the offending car back in order to hold your lane? Or does it avoid the collision and run you off the road? Neiter of those. The first and foremost rule in traffic is to act predicatable. And the only predictable reaction to something blocking your way is to BREAK! Seriously, what is it always with people that makes them think that a change in direction (swerving) was a good collision avoidcance strategy? Newsflash, it's not! When swerving you don't change your momentum, you just redirect it (apart from friction losses, of course, the the teensy bit of angular momentum transferred to the planet). More importantly, swerving combined with braking may cause loss of control, which is where there are Anti Break-lock Systems. But ABS will increase your stopping distance, so modern ABS will kick only if the steering wheel does not point into the forward direction. Of all the scenarios thrown at self driving cars ethics, none of them is an issue if basic traffic rules are followed. And the first and foremost one is, that you always keep so much distance in front of you, that you can come to a full stop within that distance by breaking. And the other first and foremost rule is you never drive faster than road conditions, and more important visibility allow. TL;DR: Don't speed and Don't tailgate! Any suddenly appearing obstacle in a vehicles way poses the same problem for a human or a computer driver. The computer driver may have the shorter reaction time, but physics is physics and the best course of action remains the same: Braking!
- ghaff 10y ago
  Not always. I've been in a scenario where I swerved into an open breakdown lane because someone swerved into my lane at high speed to avoid running into the car in front (because they had slammed on their brakes after being inattentive). Lots of swerving high-speed vehicles but no one ended up colliding. But I agree as a general rule. A lot of this trolley problem stuff is way overblown. Far and away the best action in the vast majority of cases is going to be to slam on the brakes. Even if someone or something still collides, the energy is going to be reduced and therefore the consequences of the collision. I certainly don't want my car flying off the pavement to avoid hitting something on the road without a really good model of why that's a safer approach for me.
wageslave420 10y ago
the car would know where its at with a GPS right? Why not just have different behavior for different places. Say on the mountain, it will try to pull to the side of the road away from the edge.

marcosdumay 10y ago

It does not run off the road... And it pushes the break hard. What kind of ethical dilemma should it have on that example?

msandford 10y ago
It has 0.5 seconds until collision so no hope of stopping. Push the brake all you want, unless that car gets back in its lane, people die.
- rickycook 10y ago
  .5 seconds is both incorrect, and plenty for a car that sees and responds instantly to make a car-length braking correction.
  - WalterBright 10y ago
    A collision may not be avoidable, but every decrement to the speed will reduce its severity. It's the same idea as having crumple zones on a car body.
- Retra 10y ago
  Do you have a specific example of this situation occurring as you've described it? If so, was there an ethical dilemma when a human was in control? If not, don't worry about it.

saidajigumi 10y ago

Yes, this situation is in the realm of the plausible, which has little to do with my comment. I'm not arguing against an analysis of the decisions made by a self-driving system. That's essential. I'm arguing against applying ethical models that utilize knowledge outside the realm of the knowledge or "reasoning" capacity of the drive algorithms. To continue your example, I'll pose that any system with unilateral behavior of self-endangerment is extremely problematic. Motivating example: assume that self-driving cars will always avoid head-on collisions with other vehicles, even if there's no safe alternative. Including driving off a cliff. Attack: manual driver drives down the wrong side of the road (or maybe the middle of a two-lane road), forcing all of the self-driving cars off the cliff during evening commute hours to a hillside residential neighborhood. I suspect all systems that don't hew to some baseline of self-preservation are subject to similar modes of attack.

schoen 10y ago
There's a whole kind of unpleasant but interesting topic about attacks against behavior of self-driving cars. I think I saw an earlier HN thread that suggested that they might put occupants at greater risk of kidnapping because the cars could refuse to take evasive actions that endanger or injure a kidnapper (or just violate a traffic law). This risk might be farfetched, but it may illustrate the concern about people knowing the limits that self-driving cars will respect and then exploiting those limits in some way.
- schoen 10y ago
  > I think I saw an earlier HN thread that suggested I might also have heard about some of these problems from Brad Templeton, who describes "Abusing robocar collision-avoidance" among many other interesting problems at http://www.templetons.com/brad/robocars/downsides.html http://www.templetons.com/brad/robocars/downsides.html I don't remember if he also mentioned the kidnapping scenario when I heard him present about this.
- imtringued 10y ago
  >I think I saw an earlier HN thread that suggested that they might put occupants at greater risk of kidnapping because the cars could refuse to take evasive actions that endanger or injure a kidnapper What alternative would a human take? Would they kill the kidnappers by driving over them?

datenwolf 10y ago

> Does your self-driving car allow the collision and push the offending car back in order to hold your lane? Or does it avoid the collision and run you off the road? Neiter of those. The first and foremost rule in traffic is to act predicatable. And the only predictable reaction to something blocking your way is to BREAK! Seriously, what is it always with people that makes them think that a change in direction (swerving) was a good collision avoidcance strategy? Newsflash, it's not! When swerving you don't change your momentum, you just redirect it (apart from friction losses, of course, the the teensy bit of angular momentum transferred to the planet). More importantly, swerving combined with braking may cause loss of control, which is where there are Anti Break-lock Systems. But ABS will increase your stopping distance, so modern ABS will kick only if the steering wheel does not point into the forward direction. Of all the scenarios thrown at self driving cars ethics, none of them is an issue if basic traffic rules are followed. And the first and foremost one is, that you always keep so much distance in front of you, that you can come to a full stop within that distance by breaking. And the other first and foremost rule is you never drive faster than road conditions, and more important visibility allow. TL;DR: Don't speed and Don't tailgate! Any suddenly appearing obstacle in a vehicles way poses the same problem for a human or a computer driver. The computer driver may have the shorter reaction time, but physics is physics and the best course of action remains the same: Braking!

ghaff 10y ago
Not always. I've been in a scenario where I swerved into an open breakdown lane because someone swerved into my lane at high speed to avoid running into the car in front (because they had slammed on their brakes after being inattentive). Lots of swerving high-speed vehicles but no one ended up colliding. But I agree as a general rule. A lot of this trolley problem stuff is way overblown. Far and away the best action in the vast majority of cases is going to be to slam on the brakes. Even if someone or something still collides, the energy is going to be reduced and therefore the consequences of the collision. I certainly don't want my car flying off the pavement to avoid hitting something on the road without a really good model of why that's a safer approach for me.

wageslave420 10y ago

the car would know where its at with a GPS right? Why not just have different behavior for different places. Say on the mountain, it will try to pull to the side of the road away from the edge.