4 ms·

I would much rather be recognized as a pedestrian than a cone. Hitting a cone isn't great, but usually fairly harmless, the same can not be said about hitting a

I would much rather be recognized as a pedestrian than a cone. Hitting a cone isn't great, but usually fairly harmless, the same can not be said about hitting a person.

astrange 5y ago
I don't think Waymo is going to program a car to intentionally hit even a traffic cone. You can never program your AV to intentionally hit anything because you might accidentally activate the "intentionally hit" routine when you don't need to. The right answer is almost always stop and wait, and try to evade the rest of the time. (This is why the trolley problem for AVs isn't real.)
- fshbbdssbbgdd 5y ago
  A car moving at speed can’t stop on a dime. It’s possible for multiple obstacles to show up in front of the car, and it has to pick which one to run over.
  - x3n0ph3n3 5y ago
    This is a hypothetical that has spawned all sorts of discussions about the trolley problem, but it's not really going to exist in reality. These cars will never swerve to hit one target over another.
    - michaelmrose 5y ago
      It's going to have to pick between less bad situations because humans in fact have to do so. Example running over an animal to avoid an accident, getting rear ended to avoid hitting a pedestrian, taking a calculated risk of being hit by oncoming traffic on a 2 lane situation when coming across a stalled car without optimal view on the road ahead. It's going to make a decision between different risk factors and between different known costs. Imagine on a 2 lane there is a pedestrian and oncoming traffic and insufficient time to stop. Do you kill the pedestrian or risk ending up in the ditch. If you are going 60 and ending up in the ditch would likely be fatal is it ok to just smear the pedestrian? What if you can slow to 20 and then ditch? These situations won't hopefully be prevalent because a massive portion of our current issues are down to inattention or stupidity and we believe ai will eventually be much better at avoiding such problems but they are still important and interesting.
      - Dylan16807 5y ago
        > It's going to have to pick between less bad situations because humans in fact have to do so. Yeah but not trolley problem style. "Don't swerve" is a valid strategy for a car. > Example running over an animal to avoid an accident Depends on the kind of accident you have in mind, so I can't give real feedback here. > getting rear ended to avoid hitting a pedestrian Always brake to avoid hitting a pedestrian. You don't choose whether to get rear-ended and it won't be your fault either. > taking a calculated risk of being hit by oncoming traffic on a 2 lane situation when coming across a stalled car without optimal view on the road ahead. When one of the choices is "sit there and wait longer", that's definitely not a trolley problem. It either thinks going forward is sufficiently safe or it doesn't. It can wait all day for someone to intervene. > Imagine on a 2 lane there is a pedestrian and oncoming traffic and insufficient time to stop. Do you kill the pedestrian or risk ending up in the ditch. If it's programmed not to swerve, it will hit the pedestrian, which is an acceptable outcome of an unacceptable situation. > If you are going 60 and ending up in the ditch would likely be fatal is it ok to just smear the pedestrian? It's sufficiently ok when the only alternative is so reckless. Getting into that situation is the not-ok part. > What if you can slow to 20 and then ditch? If you can slow to 20 then hit the pedestrian at 20.
        988747 5y ago
        > > Example running over an animal to avoid an accident > Depends on the kind of accident you have in mind, so I can't give real feedback here. Typical scenario here is a dog appearing on your lane in a high-traffic, multi-lane road. You don't swerve, you don't even break too hard, because hitting the dog is much better outcome than hitting another car, or landing in a ditch, or getting rear ended.
        michaelmrose 5y ago
        In many situations hitting the ditch at 20 is unlikely to kill you but hitting the pedestrian will still probably kill them.
      - tbp105 5y ago
        Humans have this problem because they get distracted and have slow reaction times. Emergency breaking can typically stop a car within about 100 yards even from freeway speeds. But a distracted human might travel more than that before they even realize they should be trying to stop. Humans are also pretty bad at adjusting to conditions, so if weather makes the road slick (wet or icy) or they have limited visibility, they don't often slow down enough to compensate. Autonomous cars have none of these problems, so aren't likely to ever run in to a "can't stop in time" situation. I can't actually think of any situation where, in the event of a likely collision, the best solution isn't to just try to stop. Even if, for some reason, you can't stop quickly enough to avoid the collision, any reduction in speed will result in a reduction in the severity of collision.
  - Vecr 5y ago
    Not really, it can just slam on the breaks as hard as ABS will allow, and stay in the middle of the lane to be as predictable as possible.
    - ineedasername 5y ago
      Slamming on it's breaks can still result in hitting something. And if there are two somethings-- slam on breaks and hit one, or swerve and hit the other-- then it's going to have to make a hard choice on what gets hit.
      - astrange 5y ago
        It is not going to make that choice because it can't be ethically programmed to make choices. Otherwise, like I said, it will someday decide it's in a hard choice situation when there's no actual problem and randomly drive itself into a wall.
        xboxnolifes 5y ago
        I'd be pretty sure that the "ethical choice" between swerving to hit a cone and slamming the brakes to maybe hit a person, all else being equal, is a pretty straightforward and can be ethically programmed.
        ineedasername 5y ago
        Yes, the choice it makes won't be based on ethics, that will ultimately come down to programmers and how well they can train the ai systems to recognize and choose the option with the least human injury. Although programmers will of course try to train it to do that. So maybe the ethical choice is actually further up the chain where people make the decisions on whether or not the AI is good enough. I don't think I that will be accepted by regulators/society until it can be demonstrated that AV's are undeniably safer-- not just as good as human drivers, much much better. Right now it's newsworthy whenever a Tesla auto pilot drives into a barrier. We'll need to get to a point where not only does that not happen, but also that it becomes newsworthy every time an AV saves a life-- swerving out if the way faster than any human reflexes could have reacted to avoid an accident, or a child chasing a ball into the street, etc.
  - astrange 5y ago
    If it has to keep moving for any reason, its goal needs to be either heading away from everything or predictability. There's no excuse for "try to hit something" as a goal unless you're a missile.
iamtedd 5y ago
It's not the cone, it's what the cone represents. Are there workers nearby? Is there a downed tree or powerline? Is the road unfinished? Is there a big sinkhole ahead?

I would much rather be recognized as a pedestrian than a cone. Hitting a cone isn't great, but usually fairly harmless, the same can not be said about hitting a person.

astrange 5y ago

I don't think Waymo is going to program a car to intentionally hit even a traffic cone. You can never program your AV to intentionally hit anything because you might accidentally activate the "intentionally hit" routine when you don't need to. The right answer is almost always stop and wait, and try to evade the rest of the time. (This is why the trolley problem for AVs isn't real.)

fshbbdssbbgdd 5y ago
A car moving at speed can’t stop on a dime. It’s possible for multiple obstacles to show up in front of the car, and it has to pick which one to run over.
- x3n0ph3n3 5y ago
  This is a hypothetical that has spawned all sorts of discussions about the trolley problem, but it's not really going to exist in reality. These cars will never swerve to hit one target over another.
  - michaelmrose 5y ago
    It's going to have to pick between less bad situations because humans in fact have to do so. Example running over an animal to avoid an accident, getting rear ended to avoid hitting a pedestrian, taking a calculated risk of being hit by oncoming traffic on a 2 lane situation when coming across a stalled car without optimal view on the road ahead. It's going to make a decision between different risk factors and between different known costs. Imagine on a 2 lane there is a pedestrian and oncoming traffic and insufficient time to stop. Do you kill the pedestrian or risk ending up in the ditch. If you are going 60 and ending up in the ditch would likely be fatal is it ok to just smear the pedestrian? What if you can slow to 20 and then ditch? These situations won't hopefully be prevalent because a massive portion of our current issues are down to inattention or stupidity and we believe ai will eventually be much better at avoiding such problems but they are still important and interesting.
    - Dylan16807 5y ago
      > It's going to have to pick between less bad situations because humans in fact have to do so. Yeah but not trolley problem style. "Don't swerve" is a valid strategy for a car. > Example running over an animal to avoid an accident Depends on the kind of accident you have in mind, so I can't give real feedback here. > getting rear ended to avoid hitting a pedestrian Always brake to avoid hitting a pedestrian. You don't choose whether to get rear-ended and it won't be your fault either. > taking a calculated risk of being hit by oncoming traffic on a 2 lane situation when coming across a stalled car without optimal view on the road ahead. When one of the choices is "sit there and wait longer", that's definitely not a trolley problem. It either thinks going forward is sufficiently safe or it doesn't. It can wait all day for someone to intervene. > Imagine on a 2 lane there is a pedestrian and oncoming traffic and insufficient time to stop. Do you kill the pedestrian or risk ending up in the ditch. If it's programmed not to swerve, it will hit the pedestrian, which is an acceptable outcome of an unacceptable situation. > If you are going 60 and ending up in the ditch would likely be fatal is it ok to just smear the pedestrian? It's sufficiently ok when the only alternative is so reckless. Getting into that situation is the not-ok part. > What if you can slow to 20 and then ditch? If you can slow to 20 then hit the pedestrian at 20.
      - 988747 5y ago
        > > Example running over an animal to avoid an accident > Depends on the kind of accident you have in mind, so I can't give real feedback here. Typical scenario here is a dog appearing on your lane in a high-traffic, multi-lane road. You don't swerve, you don't even break too hard, because hitting the dog is much better outcome than hitting another car, or landing in a ditch, or getting rear ended.
      - michaelmrose 5y ago
        In many situations hitting the ditch at 20 is unlikely to kill you but hitting the pedestrian will still probably kill them.
    - tbp105 5y ago
      Humans have this problem because they get distracted and have slow reaction times. Emergency breaking can typically stop a car within about 100 yards even from freeway speeds. But a distracted human might travel more than that before they even realize they should be trying to stop. Humans are also pretty bad at adjusting to conditions, so if weather makes the road slick (wet or icy) or they have limited visibility, they don't often slow down enough to compensate. Autonomous cars have none of these problems, so aren't likely to ever run in to a "can't stop in time" situation. I can't actually think of any situation where, in the event of a likely collision, the best solution isn't to just try to stop. Even if, for some reason, you can't stop quickly enough to avoid the collision, any reduction in speed will result in a reduction in the severity of collision.
- Vecr 5y ago
  Not really, it can just slam on the breaks as hard as ABS will allow, and stay in the middle of the lane to be as predictable as possible.
  - ineedasername 5y ago
    Slamming on it's breaks can still result in hitting something. And if there are two somethings-- slam on breaks and hit one, or swerve and hit the other-- then it's going to have to make a hard choice on what gets hit.
    - astrange 5y ago
      It is not going to make that choice because it can't be ethically programmed to make choices. Otherwise, like I said, it will someday decide it's in a hard choice situation when there's no actual problem and randomly drive itself into a wall.
      - xboxnolifes 5y ago
        I'd be pretty sure that the "ethical choice" between swerving to hit a cone and slamming the brakes to maybe hit a person, all else being equal, is a pretty straightforward and can be ethically programmed.
      - ineedasername 5y ago
        Yes, the choice it makes won't be based on ethics, that will ultimately come down to programmers and how well they can train the ai systems to recognize and choose the option with the least human injury. Although programmers will of course try to train it to do that. So maybe the ethical choice is actually further up the chain where people make the decisions on whether or not the AI is good enough. I don't think I that will be accepted by regulators/society until it can be demonstrated that AV's are undeniably safer-- not just as good as human drivers, much much better. Right now it's newsworthy whenever a Tesla auto pilot drives into a barrier. We'll need to get to a point where not only does that not happen, but also that it becomes newsworthy every time an AV saves a life-- swerving out if the way faster than any human reflexes could have reacted to avoid an accident, or a child chasing a ball into the street, etc.
- astrange 5y ago
  If it has to keep moving for any reason, its goal needs to be either heading away from everything or predictability. There's no excuse for "try to hit something" as a goal unless you're a missile.

iamtedd 5y ago

It's not the cone, it's what the cone represents. Are there workers nearby? Is there a downed tree or powerline? Is the road unfinished? Is there a big sinkhole ahead?