3 ms·

This is an important step for testing, and I'm surprised that this is being spun as a move towards commercialization. As of right now, waymo has only been doin

This is an important step for testing, and I'm surprised that this is being spun as a move towards commercialization. As of right now, waymo has only been doing what could metaphorically be called unit testing. That is, they test the cars behavior in very controlled but unrealistic environments, looking for very specific responses. The accident rate that they've incurred is likely ridiculously skewed: they've been driving in good weather, on meaningless routes (not chosen by destination, but by route features), at relatively safe times of day, at slow speeds, and they've been doing it extremely cautiously with engineers ready to take over in a moments notice. This is exactly what they should have been doing, but politically it is misleading. Most human drivers, given those same constraints, would also do extremely well and way better than average. They've done well, but we have little basis for comparison with the average driver. This is the first step towards integration testing. They get to see how the cars behavior integrates together across various scenarios that are much closer to real life. They are driving on actual routes that real people travel on...routes that aren't chosen in order to test a specific behavior. Accident rates are going to go up. That's a good thing...its a move towards the things humans find more difficult too. We should, however, expect slowing progress to level 4 autonomy. This is typical of system capability growth; exponential in the beginning, asymptotic near the end. People that are rushing this are out of line; akin to immediate commercializations of lab rat successes. Give them time.

teach 10y ago
> they test the cars in very controlled but unrealistic environments Are you sure about this? Google has had self-driving cars in Austin for years and I'm pretty sure there's never a good time to be driving here.
- saosebastiao 10y ago
  Pretty damn sure. Even if an engineer drives his normal car to work, hops in a waymo car, and drives 9-5, and then drives a normal car back home, they've already constructed a scenario that is heavily time-weighted towards lighter than human-representative traffic experiences. Even small biases that you don't even think are meaningful can have a big impact on outcomes. But it doesn't stop there. Google hasn't tested snow at all. I'd be surprised if their testing locations represented rainy weather that is representative for more than 10% of the US. They don't test freeways, and by extension, do not test onramps and offramps. In California, they don't even go over 25 mph. Under those conditions they have been ostensibly pretty good at not causing accidents, but how good are they at avoiding accidents that are not their fault? This means accidents caused by other drivers and acts of god. Safety is far more than just causality. But more importantly, their miles traveled very easily exclude small snippets of time where an engineer takes over, but still counts the rest of the journey where the computer did fine. How representative is that? How good are humans when we let jesus take the wheel the moment driving gets slightly complicated? If a human overrides the computer only 1% of the time, that could still be anywhere from 1% to 100% of trips with at least one human override. It's very easy to catastrophically mislead yourself when working with averages [0]. Statisticians and Economists go through absolutely absurd lengths to make comparisons robust. Even if we had access to 100% of the data on Waymo's self driving cars, I don't know a single statistician that would be comfortable making a comparison to extremely coarsely aggregated data on human driving. Only through extensive testing in situations that are representative of how humans (and not just test engineers) drive will we be able to conclude that one is safer than the other. And that's why Phoenix is important to Waymo. [0] https://www.google.com/search?client=ubuntu&channel=fs&q=the+problem+with+averages&ie=utf-8&oe=utf-8 https://www.google.com/search?client=ubuntu&channel=fs&q=the...

teach 10y ago

> they test the cars in very controlled but unrealistic environments Are you sure about this? Google has had self-driving cars in Austin for years and I'm pretty sure there's never a good time to be driving here.

saosebastiao 10y ago
Pretty damn sure. Even if an engineer drives his normal car to work, hops in a waymo car, and drives 9-5, and then drives a normal car back home, they've already constructed a scenario that is heavily time-weighted towards lighter than human-representative traffic experiences. Even small biases that you don't even think are meaningful can have a big impact on outcomes. But it doesn't stop there. Google hasn't tested snow at all. I'd be surprised if their testing locations represented rainy weather that is representative for more than 10% of the US. They don't test freeways, and by extension, do not test onramps and offramps. In California, they don't even go over 25 mph. Under those conditions they have been ostensibly pretty good at not causing accidents, but how good are they at avoiding accidents that are not their fault? This means accidents caused by other drivers and acts of god. Safety is far more than just causality. But more importantly, their miles traveled very easily exclude small snippets of time where an engineer takes over, but still counts the rest of the journey where the computer did fine. How representative is that? How good are humans when we let jesus take the wheel the moment driving gets slightly complicated? If a human overrides the computer only 1% of the time, that could still be anywhere from 1% to 100% of trips with at least one human override. It's very easy to catastrophically mislead yourself when working with averages [0]. Statisticians and Economists go through absolutely absurd lengths to make comparisons robust. Even if we had access to 100% of the data on Waymo's self driving cars, I don't know a single statistician that would be comfortable making a comparison to extremely coarsely aggregated data on human driving. Only through extensive testing in situations that are representative of how humans (and not just test engineers) drive will we be able to conclude that one is safer than the other. And that's why Phoenix is important to Waymo. [0] https://www.google.com/search?client=ubuntu&channel=fs&q=the+problem+with+averages&ie=utf-8&oe=utf-8 https://www.google.com/search?client=ubuntu&channel=fs&q=the...