4 ms·

I'm disappointed the article wasn't a bit more skeptical of some of the claims. Certainly the simulation-based testing is a good thing, but stats about how many

I'm disappointed the article wasn't a bit more skeptical of some of the claims. Certainly the simulation-based testing is a good thing, but stats about how many billions of simulated miles have been driven can create a self-reinforcing delusion if everyone involved isn't careful to remember that the simulations can only work with well-known and expected situations. It sounds like Waymo realizes this and is building a huge library of scenarios to evaluate, but one million simulated miles are not worth a hundred real miles in terms of confidence in the system, and that is not the message this article portrays. Overall this article does give hints though that autonomous vehicles are truly much further away than anyone wants to admit. The story about being flummoxed by multilane roundabouts is depressing. If they didn't know such things existed then they were incredibly sloppy in their data gathering. If they truly thought scaling from a simple roundabout to a multilane one would be trivial then their staff don't have the right mindset for this problem space. Also note that all the testing is happening in flat, desert landscapes where there are no weather or lighting challenges. Their pictured model residential street only has stubs of driveways. No houses, no trees. I'm sure they know these are gaps but I worry they underestimate the challenges of adapting to entirely different driving environments. Especially when machine learning is involved and you're simulating 99% of your mileage... Looking forward to checking back in 2040 though.

ghshephard 9y ago
"One million simulated miles are not worth a hundred real miles in terms of confidence in the system" I'm wondering why you say that? Wouldn't the more relevant factor be the type of miles, regardless of whether they are simulated or real? For example, 1 mile of simulated "Interesting Scenarios" (Duck on Street, Bicyclist going wrong way, pedestrian running onto road) is likely worth 100,000 miles of normal Freeway driving.
bluGill 9y ago
One millions miles of different situations is worth more than 100 miles anywhere. Even your most carefully chosen worst case route won't cover all situations. To get close you have to pay people to send their kid running between parked cars into the street at just the right moment. Nobody sane will put a real kids into that situation, but the simulation can (should!) have thousands of different variations. The simulation can afford to run all of those situations regularly. Simulation can ensure it is raining when you need it. You need miles of real driving too, but they are too expensive and unpredictable to rely on for more than a final check.
bullfightonmars 9y ago
> The story about being flummoxed by multilane roundabouts is depressing Don't underestimate the bizarre engineering of infrastructure in Texas. Every road and intersection is bespoke, unlike any other you have previously experienced. It is quite difficult to navigate Austin as a human driver, I suspect it makes for a good testing ground for Waymo.

ghshephard 9y ago

"One million simulated miles are not worth a hundred real miles in terms of confidence in the system" I'm wondering why you say that? Wouldn't the more relevant factor be the type of miles, regardless of whether they are simulated or real? For example, 1 mile of simulated "Interesting Scenarios" (Duck on Street, Bicyclist going wrong way, pedestrian running onto road) is likely worth 100,000 miles of normal Freeway driving.

bluGill 9y ago

One millions miles of different situations is worth more than 100 miles anywhere. Even your most carefully chosen worst case route won't cover all situations. To get close you have to pay people to send their kid running between parked cars into the street at just the right moment. Nobody sane will put a real kids into that situation, but the simulation can (should!) have thousands of different variations. The simulation can afford to run all of those situations regularly. Simulation can ensure it is raining when you need it. You need miles of real driving too, but they are too expensive and unpredictable to rely on for more than a final check.

bullfightonmars 9y ago

> The story about being flummoxed by multilane roundabouts is depressing Don't underestimate the bizarre engineering of infrastructure in Texas. Every road and intersection is bespoke, unlike any other you have previously experienced. It is quite difficult to navigate Austin as a human driver, I suspect it makes for a good testing ground for Waymo.