3 ms·

At 30:18 there's a very interesting tidbit. https://youtu.be/zdKCQKBvH-A?t=1818 https://youtu.be/zdKCQKBvH-A?t=1818 They way I understand that is the route pl

by 22c 5y ago

At 30:18 there's a very interesting tidbit. https://youtu.be/zdKCQKBvH-A?t=1818 https://youtu.be/zdKCQKBvH-A?t=1818 They way I understand that is the route planner, which I assume is ran completely separately to the self-driving car, removes un-navigable spaces from the route planning. So in this situation, the roadside assistance driver is surprised that the entire lane has not been removed from the route planner. That's pretty damning and it makes me wonder if there is a huge gap in autonomy here that is being papered over by clever route planning (ie. avoid that lane altogether and the AI won't have to worry about the cones). The AI should be able to figure out on its own that the right lane was closed, it shouldn't need route planning to do that for it.

sounds 5y ago
This is clearly showing how Waymo software is not a full Artificial General Intelligence - something that can reason on its own, like a fully awake, rested human. Take the moment when the car first gets stuck. It attempts a right turn, spots the cones just fine, and correctly makes the assessment that it can avoid the cones and proceed in the right lane after the turn. As a human, our thinking process does not stop there. We notice that the traffic already flowing on the street is _not_ using the right hand lane. We notice that the cones are vaguely in a merge pattern to indicate that cars entering from the side street should merge left - if you were looking at the cones from a vantage point behind where the cones are placed, and if you can call the first two cones enough of a "please merge left" pattern. The Waymo car can't see that, though. It's located sort of off to one side, attempting to reason about it from a high angle. And two cones from this angle could be a lot of things, and it's not 100% clear the cones close off the right lane. One theory might be that the Waymo model has been trained on a lot of situations but not this specific situation - something about the high angle, something about the position of the cones, something about the route plan managed to find a loophole in the AI training set and produce a weird result. That's just the limitations of the current state-of-the-art Machine Learning. Find a glaring incorrect answer? Retrain with more training data. Part of what makes full Artificial General Intelligence (AGI) with deductive reasoning can retrain itself without making mistakes along the way, unless something incomprehensible happens. And I use the word "incomprehensible" here to point straight at AGI's ability to _comprehend_ the situation at a much higher level than current machine learning techniques. tl;dr: The Waymo Car didn't get it. And that's about what I would expect. The real gem is that the _humans_ in the assistance team were fighting each other. The remote control operator kept jamming the "unstick yourself" button even as the roadside assistance guys were walking up to take over. The real danger, I think, is more of this in the future: the remote operators defeating all the safeties. Since the remote operator is not at any real risk of personal harm, what do they care?
- jacoblambda 5y ago
  The remote operator wasn't "unjamming" the vehicle FYI. Rather the car kept trying to unstuck itself until the in person operator unlocked the car doors. IMO it's more concerning that the vehicle keeps trying after it gets into a "call roadside assistance" mode rather than staying still or allowing the remote operator to assume manual control (say to move the vehicle out of the lane).
  - toast0 5y ago
    Yeah, I would think you would want to have someone confirm before the vehicle starts driving again. Especially in the case where the assistance driver is preparing to enter the vehicle. Perhaps that did happen behind the scenes; I hope Waymo will publish an incident report with more details; their statement did say the Fleet Response team provided incorrect guidance, so maybe. Also, it really seemed like the Rider Support agent didn't have complete information on the status of the car or any apparent ability to intervene. I would love to know about how incident response is organized behind the scenes; there wasn't a lot of communication to the occupant about what was happening, which isn't ideal. The Roadside assistance request apparently being dropped when the vehicle began moving again, but before it demonstrated it was able to navigate the conditions is a bit concerning as well. It makes the response feel disorganized, although again, it could just be lack of information passing to the occupant.
- babesh 5y ago
  Remember that optical illusion that was either a woman’s neck or a swan? I think it’s the same thing here. The flow of traffic recast our perception of the sequence of cones as blocking the right lane instead of blocking the left lane or both lanes. I wonder how code/neural networks can be coded to push the perception a certain way using signals/logic outside the immediate code/network.
machinelearning 5y ago
I think the only reasonable path forward is to move away from completely general AI (because we all know that anyone who claims they have reliable general AI is lying) and embrace specific AI. Intelligent route planning to compensate for the deficits of our current perception tech stack is a good thing and allows greater reliability. It is harder to scale this but it’s a far more tractable problem than aiming for complete generalization.
- slimsag 5y ago
  This is literally what Waymo is already doing.
  - machinelearning 5y ago
    This is literally what I was saying. It was in response to: “That's pretty damning and it makes me wonder if there is a huge gap in autonomy here that is being papered over by clever route planning” To which I said that clever route planning is not a hack but the only reasonable way forward.
    - slimsag 5y ago
      I didn't mean it in a derogatory way. I was just trying to say it in even more plain terms - because this whole thread was full of people who seemingly have no idea about how AVs are being approached today and are, as is usual for HN, suggesting they are smarter than the actual people working on the technology.
- joe_the_user 5y ago
  Waymo has never claimed completely general AI. Indeed, no serious researcher or company claims such a thing. The term "AI" has gained currency in the last few years but it's just marketing or shorthand for machine learn usually based on deep neural networks. Intelligent route planning to compensate for the deficits of our current perception tech stack is a good thing and allows greater reliability. This is what Waymo is doing and has done from the start. But this approach itself has significant limitations as the video shows. You have to automate the route planning process also, remember.

by 22c 5y ago

sounds 5y ago

This is clearly showing how Waymo software is not a full Artificial General Intelligence - something that can reason on its own, like a fully awake, rested human. Take the moment when the car first gets stuck. It attempts a right turn, spots the cones just fine, and correctly makes the assessment that it can avoid the cones and proceed in the right lane after the turn. As a human, our thinking process does not stop there. We notice that the traffic already flowing on the street is _not_ using the right hand lane. We notice that the cones are vaguely in a merge pattern to indicate that cars entering from the side street should merge left - if you were looking at the cones from a vantage point behind where the cones are placed, and if you can call the first two cones enough of a "please merge left" pattern. The Waymo car can't see that, though. It's located sort of off to one side, attempting to reason about it from a high angle. And two cones from this angle could be a lot of things, and it's not 100% clear the cones close off the right lane. One theory might be that the Waymo model has been trained on a lot of situations but not this specific situation - something about the high angle, something about the position of the cones, something about the route plan managed to find a loophole in the AI training set and produce a weird result. That's just the limitations of the current state-of-the-art Machine Learning. Find a glaring incorrect answer? Retrain with more training data. Part of what makes full Artificial General Intelligence (AGI) with deductive reasoning can retrain itself without making mistakes along the way, unless something incomprehensible happens. And I use the word "incomprehensible" here to point straight at AGI's ability to _comprehend_ the situation at a much higher level than current machine learning techniques. tl;dr: The Waymo Car didn't get it. And that's about what I would expect. The real gem is that the _humans_ in the assistance team were fighting each other. The remote control operator kept jamming the "unstick yourself" button even as the roadside assistance guys were walking up to take over. The real danger, I think, is more of this in the future: the remote operators defeating all the safeties. Since the remote operator is not at any real risk of personal harm, what do they care?

jacoblambda 5y ago
The remote operator wasn't "unjamming" the vehicle FYI. Rather the car kept trying to unstuck itself until the in person operator unlocked the car doors. IMO it's more concerning that the vehicle keeps trying after it gets into a "call roadside assistance" mode rather than staying still or allowing the remote operator to assume manual control (say to move the vehicle out of the lane).
- toast0 5y ago
  Yeah, I would think you would want to have someone confirm before the vehicle starts driving again. Especially in the case where the assistance driver is preparing to enter the vehicle. Perhaps that did happen behind the scenes; I hope Waymo will publish an incident report with more details; their statement did say the Fleet Response team provided incorrect guidance, so maybe. Also, it really seemed like the Rider Support agent didn't have complete information on the status of the car or any apparent ability to intervene. I would love to know about how incident response is organized behind the scenes; there wasn't a lot of communication to the occupant about what was happening, which isn't ideal. The Roadside assistance request apparently being dropped when the vehicle began moving again, but before it demonstrated it was able to navigate the conditions is a bit concerning as well. It makes the response feel disorganized, although again, it could just be lack of information passing to the occupant.
babesh 5y ago
Remember that optical illusion that was either a woman’s neck or a swan? I think it’s the same thing here. The flow of traffic recast our perception of the sequence of cones as blocking the right lane instead of blocking the left lane or both lanes. I wonder how code/neural networks can be coded to push the perception a certain way using signals/logic outside the immediate code/network.

machinelearning 5y ago

I think the only reasonable path forward is to move away from completely general AI (because we all know that anyone who claims they have reliable general AI is lying) and embrace specific AI. Intelligent route planning to compensate for the deficits of our current perception tech stack is a good thing and allows greater reliability. It is harder to scale this but it’s a far more tractable problem than aiming for complete generalization.

slimsag 5y ago
This is literally what Waymo is already doing.
- machinelearning 5y ago
  This is literally what I was saying. It was in response to: “That's pretty damning and it makes me wonder if there is a huge gap in autonomy here that is being papered over by clever route planning” To which I said that clever route planning is not a hack but the only reasonable way forward.
  - slimsag 5y ago
    I didn't mean it in a derogatory way. I was just trying to say it in even more plain terms - because this whole thread was full of people who seemingly have no idea about how AVs are being approached today and are, as is usual for HN, suggesting they are smarter than the actual people working on the technology.
joe_the_user 5y ago
Waymo has never claimed completely general AI. Indeed, no serious researcher or company claims such a thing. The term "AI" has gained currency in the last few years but it's just marketing or shorthand for machine learn usually based on deep neural networks. Intelligent route planning to compensate for the deficits of our current perception tech stack is a good thing and allows greater reliability. This is what Waymo is doing and has done from the start. But this approach itself has significant limitations as the video shows. You have to automate the route planning process also, remember.