4 ms·

> The most common problem, especially with Waymo, is being rear-ended while cautiously entering an intersection with limited visibility. Their system will start

by zawerf 8y ago

> The most common problem, especially with Waymo, is being rear-ended while cautiously entering an intersection with limited visibility. Their system will start forward to get a better view, then detect cross traffic and stop This actually highlights a really hard problem that (perfect) self driving will need to solve: theory of mind. https://en.wikipedia.org/wiki/Theory_of_mind https://en.wikipedia.org/wiki/Theory_of_mind Not only does the AI need to empathize and predict how typical humans react, they also need to be easy for humans to empathize with. Humans typically use themselves as a template for a reasonable range of reactions. So when the capabilities don't match up (like needing to scoot up to process how to cross where a typical human wouldn't need to), it will violate their expectation of you. So in this case being cautious is considered a crazy and erratic behavior to a human. (e.g., "what were you thinking?! why did you stop!? shouldn't you've seen that the road was empty?!")

Analemma_ 8y ago
Yeah, this is a good point. Most of the HN crowd understands that ML-powered autonomous vehicles will have crazy failure modes that no human would ever do. Proponents of self-driving cars claim that the reverse is also true and that, on average, self-driving cars will be safer on average than humans – and that might be true! But it still matters that the humans remaining on the road can't predict what these things will do and our theory of mind won't work: even bad drivers are usually bad in predictable ways, but a software system that makes unexpected mistakes humans can't understand has second-order dangers on the road besides the ones it creates itself.
ScottBurson 8y ago
The only time I've ever been rear-ended was that way — pulling out of a parking lot where the street was at a slightly higher level, so I couldn't see around the parked cars on either side of the exit. I started to pull out rather quickly, but then was forced to hit the brakes, and the person behind me — well, I don't know where they were looking. I guess that's not quite the same scenario, though, since the person behind me couldn't see the cross traffic either; they were just cueing off my own behavior to conclude that it must be clear. Anyway, you're right: good driving isn't just predicting what other people are going to do; it's also making sure your own behavior gives them the correct idea about your intentions.
Izkata 8y ago
> Not only does the AI need to empathize and predict how typical humans react, they also need to be easy for humans to empathize with. Humans typically use themselves as a template for a reasonable range of reactions. Hm, this past season of Westworld kinda touched on how this very particular thought can easily go wrong. Trying to avoid spoilers, one of the programmers commented on how the teeny tiniest tweak to parameters created Hosts that would [motions to what they were watching, a normally-benign personality killing townspeople for fun]. The awake Host with him commented "it's gone insane". His response was something along the lines of, "in the range of possible actions, what humans call 'sane' only exists in a very narrow band".

by zawerf 8y ago

Analemma_ 8y ago

Yeah, this is a good point. Most of the HN crowd understands that ML-powered autonomous vehicles will have crazy failure modes that no human would ever do. Proponents of self-driving cars claim that the reverse is also true and that, on average, self-driving cars will be safer on average than humans – and that might be true! But it still matters that the humans remaining on the road can't predict what these things will do and our theory of mind won't work: even bad drivers are usually bad in predictable ways, but a software system that makes unexpected mistakes humans can't understand has second-order dangers on the road besides the ones it creates itself.

ScottBurson 8y ago

The only time I've ever been rear-ended was that way — pulling out of a parking lot where the street was at a slightly higher level, so I couldn't see around the parked cars on either side of the exit. I started to pull out rather quickly, but then was forced to hit the brakes, and the person behind me — well, I don't know where they were looking. I guess that's not quite the same scenario, though, since the person behind me couldn't see the cross traffic either; they were just cueing off my own behavior to conclude that it must be clear. Anyway, you're right: good driving isn't just predicting what other people are going to do; it's also making sure your own behavior gives them the correct idea about your intentions.

Izkata 8y ago

> Not only does the AI need to empathize and predict how typical humans react, they also need to be easy for humans to empathize with. Humans typically use themselves as a template for a reasonable range of reactions. Hm, this past season of Westworld kinda touched on how this very particular thought can easily go wrong. Trying to avoid spoilers, one of the programmers commented on how the teeny tiniest tweak to parameters created Hosts that would [motions to what they were watching, a normally-benign personality killing townspeople for fun]. The awake Host with him commented "it's gone insane". His response was something along the lines of, "in the range of possible actions, what humans call 'sane' only exists in a very narrow band".