4 ms·
This is clearly showing how Waymo software is not a full Artificial General Intelligence - something that can reason on its own, like a fully awake, rested huma
by sounds 5y ago
This is clearly showing how Waymo software is not a full Artificial General Intelligence - something that can reason on its own, like a fully awake, rested human.
Take the moment when the car first gets stuck. It attempts a right turn, spots the cones just fine, and correctly makes the assessment that it can avoid the cones and proceed in the right lane after the turn.
As a human, our thinking process does not stop there. We notice that the traffic already flowing on the street is _not_ using the right hand lane. We notice that the cones are vaguely in a merge pattern to indicate that cars entering from the side street should merge left - if you were looking at the cones from a vantage point behind where the cones are placed, and if you can call the first two cones enough of a "please merge left" pattern.
The Waymo car can't see that, though. It's located sort of off to one side, attempting to reason about it from a high angle. And two cones from this angle could be a lot of things, and it's not 100% clear the cones close off the right lane.
One theory might be that the Waymo model has been trained on a lot of situations but not this specific situation - something about the high angle, something about the position of the cones, something about the route plan managed to find a loophole in the AI training set and produce a weird result.
That's just the limitations of the current state-of-the-art Machine Learning. Find a glaring incorrect answer? Retrain with more training data.
Part of what makes full Artificial General Intelligence (AGI) with deductive reasoning can retrain itself without making mistakes along the way, unless something incomprehensible happens. And I use the word "incomprehensible" here to point straight at AGI's ability to _comprehend_ the situation at a much higher level than current machine learning techniques.
tl;dr: The Waymo Car didn't get it. And that's about what I would expect.
The real gem is that the _humans_ in the assistance team were fighting each other. The remote control operator kept jamming the "unstick yourself" button even as the roadside assistance guys were walking up to take over. The real danger, I think, is more of this in the future: the remote operators defeating all the safeties. Since the remote operator is not at any real risk of personal harm, what do they care?
- jacoblambda 5y agoThe remote operator wasn't "unjamming" the vehicle FYI. Rather the car kept trying to unstuck itself until the in person operator unlocked the car doors. IMO it's more concerning that the vehicle keeps trying after it gets into a "call roadside assistance" mode rather than staying still or allowing the remote operator to assume manual control (say to move the vehicle out of the lane).
- toast0 5y agoYeah, I would think you would want to have someone confirm before the vehicle starts driving again. Especially in the case where the assistance driver is preparing to enter the vehicle. Perhaps that did happen behind the scenes; I hope Waymo will publish an incident report with more details; their statement did say the Fleet Response team provided incorrect guidance, so maybe. Also, it really seemed like the Rider Support agent didn't have complete information on the status of the car or any apparent ability to intervene. I would love to know about how incident response is organized behind the scenes; there wasn't a lot of communication to the occupant about what was happening, which isn't ideal. The Roadside assistance request apparently being dropped when the vehicle began moving again, but before it demonstrated it was able to navigate the conditions is a bit concerning as well. It makes the response feel disorganized, although again, it could just be lack of information passing to the occupant.
- babesh 5y agoRemember that optical illusion that was either a woman’s neck or a swan? I think it’s the same thing here. The flow of traffic recast our perception of the sequence of cones as blocking the right lane instead of blocking the left lane or both lanes. I wonder how code/neural networks can be coded to push the perception a certain way using signals/logic outside the immediate code/network.