6 ms·

Could be a bit more involved than that. The Machine acted in a way that was not expected by the human driver. This gets to a fundamental issue we will have in

Could be a bit more involved than that. The Machine acted in a way that was not expected by the human driver. This gets to a fundamental issue we will have in the transition to the utopian driverless future. Learning to identify and react to this new class of driver.

mannykannot 9y ago
I am not sure that the truck driver had any expectations, as he acted as if he had not seen the shuttle. The article sneers that "surely this situation is not so rare that the shuttle’s designers did not allow for it?", but of course it is not that simple - there are a great many cases in everyday driving where the projected motion of the vehicles would lead to a collision if nothing was done to stop it, and so it is clearly infeasible to react to all such situations as if nothing else will be done. The passengers of the shuttle recognized the impending collision because they understood that the truck driver had probably not seen the shuttle. Approximating these applications of our theory of mind is probably the hardest part about automated driving.
spyspy 9y ago
Really moving the goal post here. Apparently _drive safely_ is an "unexpected" behavior.
- agumonkey 9y ago
  next: following the law should be illegal
- jacobush 9y ago
  Was it driving safely, then?
  - michaelbuckbee 9y ago
    The shuttle was literally not moving and was struck by the human controlled truck.
    - Cthulhu_ 9y ago
      Safety can also mean avoiding / swerving / reversing. I've had motorcycle lessons, one factor in those things is that their braking power is less than that of a car (less wheels, less stability, etc), so besides the emergency stop, you also learn how to do an emergency swerve - using the motorcycle's superior agility to avoid an accident. Just an anecdote. In this case, damage would've been avoided if the car had a horn and/or was more visible and/or had backed up and/or the truck had sensors.
    - mtgx 9y ago
      Isn't it illegal to do stuff like "blocking an intersection" for instance? What will a self-driving car do if for instance someone is driving in the opposite direction, towards it? Will it just continue driving, freeze, or get out of the way? Humans would likely get out of the way, even if that means "breaking the law" by going off-road or whatever. A self-driving car probably wouldn't do that, thus it would be the kind of "unexpected driving" that could really throw humans off, and may even cause more accidents than humans, in certain specific situations.
      - Piskvorrr 9y ago
        In such situations, an illegal action of another might force you into illegal behavior in any case: swerve, get out of the way into opposite direction, stop and block the intersection? (How is that causing an accident, if it's reactive towards another's reckless driving?)
      - jacobush 9y ago
        Also, there are even deeper levels here. Going off-road in such a situation may not actually be against the law. It may look like it on the books, but when such a case is interpreted by judges and peers, they will certainly in some cases decide that swerving or going off-road was perfectly legal, for such a situation.
        mr_toad 9y ago
        Judges might forgive a human who drove onto a sidewalk to avoid an accident, but they're not likely to be as forgiving of a manufacturer that programmed their cars to do the same thing.
jdavis703 9y ago
I'm sure you've interacted with truckers before. They drive in a manner of "biggest object wins" (not all, but many, especially city delivery drivers). It's pretty common for cars, bikes and people to yield to trucks doing "strange" movements, even when the trucker doesn't have the right of way.
- binarymax 9y ago
  Unrelated but interestingly, it's exactly the opposite at sea. More inertia makes it harder to react in a timely manner, smaller craft must give way.
  - joncampbelldev 9y ago
    What you describe is exactly the same as what jdavis703 said, not the opposite.
    - binarymax 9y ago
      Rules of the road give less powered vehicles the right of way. trucks should yield to bicycles and pedestrians. Trucks flout these rules regularly. --EDIT-- it seems my comments are being misunderstood. I am simply stating the law as it exists, and the orthogonality between land and sea in regards to these laws. I know that Newtonian physics conflicts with the land aspect of these laws.
      - EADGBE 9y ago
        Just a reminder that trucks have more power to go, but less power to stop.
      - falcolas 9y ago
        Yes, because they have the right of weight... and strict timelines they often couldn’t meet if they did follow the rules to a tee. If a truck driver can complete a run in an hour and only infrequently have an accident by bending the rules, why would a “maximize profit” company give them two hours so they could follow the rules properly?
mikeash 9y ago
Stopping for the unexpected is pretty much expected. Lots of human drivers freeze up when confronted with a situation like this. I don’t see any lesson here other than that human drivers are atrocious and the fully autonomous future can’t come quickly enough.
- Piskvorrr 9y ago
  Ad absurdum, humans are atrocious and a humanless future can't come quickly enough. (People are jaywalking, EXTERMINATE EXTERMINATE EXTERMINATE!)
  - mikeash 9y ago
    I don't think Benderism is quite the logical conclusion from advocating for human drivers to be removed from the roads.
    - Piskvorrr 9y ago
      Nope. You said fully autonomous. The cars are not the only moving objects in existence, sorry to spoil it.
      - mikeash 9y ago
        Context!
Cthulhu_ 9y ago
In the utopian driverless future, the truck was also driverless and the two had communicated their intent minutes before the incident so it never happened in the first place.
- Piskvorrr 9y ago
  Or tried communicating but didn't, because protocol mismatch (Mars Orbiter, Gimli Glider). Oh wait!
  - kazagistar 9y ago
    In case of uncertainty they would both stop before a collision, because they both would have cameras that see all around them, rather then just backing into a blind spot.
mtgx 9y ago
> The Machine acted in a way that was not expected by the human driver. This is why I've been saying it's not enough for a self-driving car to just be better than the "average human driver". It needs to be way better, maybe 10x better. Because these cars are likely not going to drive the way we expect a human to drive in many situations, which could end-up causing more accidents due to this conflict with how humans actually drive, even if the car is "technically" as safe as an average human driver.
smsm42 9y ago
> The Machine acted in a way that was not expected by the human driver. Nope, the human driver was careless and did not look around, failing to notice the obstacle. If there was a human-driven car instead, we'd have never heard of it and the matter would have been resolved by regular insurance payments that happen every day in every city a thousand times. But since there's an automatic car involved now it is a philosophical conundrum. Nope, it's not, it's just some guy in a truck being careless.

mannykannot 9y ago

I am not sure that the truck driver had any expectations, as he acted as if he had not seen the shuttle. The article sneers that "surely this situation is not so rare that the shuttle’s designers did not allow for it?", but of course it is not that simple - there are a great many cases in everyday driving where the projected motion of the vehicles would lead to a collision if nothing was done to stop it, and so it is clearly infeasible to react to all such situations as if nothing else will be done. The passengers of the shuttle recognized the impending collision because they understood that the truck driver had probably not seen the shuttle. Approximating these applications of our theory of mind is probably the hardest part about automated driving.

spyspy 9y ago

Really moving the goal post here. Apparently _drive safely_ is an "unexpected" behavior.

agumonkey 9y ago
next: following the law should be illegal
jacobush 9y ago
Was it driving safely, then?
- michaelbuckbee 9y ago
  The shuttle was literally not moving and was struck by the human controlled truck.
  - Cthulhu_ 9y ago
    Safety can also mean avoiding / swerving / reversing. I've had motorcycle lessons, one factor in those things is that their braking power is less than that of a car (less wheels, less stability, etc), so besides the emergency stop, you also learn how to do an emergency swerve - using the motorcycle's superior agility to avoid an accident. Just an anecdote. In this case, damage would've been avoided if the car had a horn and/or was more visible and/or had backed up and/or the truck had sensors.
  - mtgx 9y ago
    Isn't it illegal to do stuff like "blocking an intersection" for instance? What will a self-driving car do if for instance someone is driving in the opposite direction, towards it? Will it just continue driving, freeze, or get out of the way? Humans would likely get out of the way, even if that means "breaking the law" by going off-road or whatever. A self-driving car probably wouldn't do that, thus it would be the kind of "unexpected driving" that could really throw humans off, and may even cause more accidents than humans, in certain specific situations.
    - Piskvorrr 9y ago
      In such situations, an illegal action of another might force you into illegal behavior in any case: swerve, get out of the way into opposite direction, stop and block the intersection? (How is that causing an accident, if it's reactive towards another's reckless driving?)
    - jacobush 9y ago
      Also, there are even deeper levels here. Going off-road in such a situation may not actually be against the law. It may look like it on the books, but when such a case is interpreted by judges and peers, they will certainly in some cases decide that swerving or going off-road was perfectly legal, for such a situation.
      - mr_toad 9y ago
        Judges might forgive a human who drove onto a sidewalk to avoid an accident, but they're not likely to be as forgiving of a manufacturer that programmed their cars to do the same thing.

jdavis703 9y ago

I'm sure you've interacted with truckers before. They drive in a manner of "biggest object wins" (not all, but many, especially city delivery drivers). It's pretty common for cars, bikes and people to yield to trucks doing "strange" movements, even when the trucker doesn't have the right of way.

binarymax 9y ago
Unrelated but interestingly, it's exactly the opposite at sea. More inertia makes it harder to react in a timely manner, smaller craft must give way.
- joncampbelldev 9y ago
  What you describe is exactly the same as what jdavis703 said, not the opposite.
  - binarymax 9y ago
    Rules of the road give less powered vehicles the right of way. trucks should yield to bicycles and pedestrians. Trucks flout these rules regularly. --EDIT-- it seems my comments are being misunderstood. I am simply stating the law as it exists, and the orthogonality between land and sea in regards to these laws. I know that Newtonian physics conflicts with the land aspect of these laws.
    - EADGBE 9y ago
      Just a reminder that trucks have more power to go, but less power to stop.
    - falcolas 9y ago
      Yes, because they have the right of weight... and strict timelines they often couldn’t meet if they did follow the rules to a tee. If a truck driver can complete a run in an hour and only infrequently have an accident by bending the rules, why would a “maximize profit” company give them two hours so they could follow the rules properly?

mikeash 9y ago

Stopping for the unexpected is pretty much expected. Lots of human drivers freeze up when confronted with a situation like this. I don’t see any lesson here other than that human drivers are atrocious and the fully autonomous future can’t come quickly enough.

Piskvorrr 9y ago
Ad absurdum, humans are atrocious and a humanless future can't come quickly enough. (People are jaywalking, EXTERMINATE EXTERMINATE EXTERMINATE!)
- mikeash 9y ago
  I don't think Benderism is quite the logical conclusion from advocating for human drivers to be removed from the roads.
  - Piskvorrr 9y ago
    Nope. You said fully autonomous. The cars are not the only moving objects in existence, sorry to spoil it.
    - mikeash 9y ago
      Context!

Cthulhu_ 9y ago

In the utopian driverless future, the truck was also driverless and the two had communicated their intent minutes before the incident so it never happened in the first place.

Piskvorrr 9y ago
Or tried communicating but didn't, because protocol mismatch (Mars Orbiter, Gimli Glider). Oh wait!
- kazagistar 9y ago
  In case of uncertainty they would both stop before a collision, because they both would have cameras that see all around them, rather then just backing into a blind spot.

mtgx 9y ago

> The Machine acted in a way that was not expected by the human driver. This is why I've been saying it's not enough for a self-driving car to just be better than the "average human driver". It needs to be way better, maybe 10x better. Because these cars are likely not going to drive the way we expect a human to drive in many situations, which could end-up causing more accidents due to this conflict with how humans actually drive, even if the car is "technically" as safe as an average human driver.

smsm42 9y ago

> The Machine acted in a way that was not expected by the human driver. Nope, the human driver was careless and did not look around, failing to notice the obstacle. If there was a human-driven car instead, we'd have never heard of it and the matter would have been resolved by regular insurance payments that happen every day in every city a thousand times. But since there's an automatic car involved now it is a philosophical conundrum. Nope, it's not, it's just some guy in a truck being careless.