4 ms·

> did we exclude drivers who were not eligible to drive One could argue here that it would be unfair to exclude the worst performing humans. Plenty of humans c

> did we exclude drivers who were not eligible to drive One could argue here that it would be unfair to exclude the worst performing humans. Plenty of humans cause accidents for easily preventable reasons, but that's an honest part of our behavior. Or to rephrase, I think excluding the categories of people who drive terribly, but do still drive nonetheless, would be less representative of reality. > Did Waymo drive across all the same areas and time periods as the humans did with an equal "proportion"? No, they didn't. Their methodology didn't adjust for things like highway vs city street miles, although it did adjust for city and state. > Could we compare "time behind wheel" instead of miles travelled? We could, but that would incentivize slow driving rather than safe driving. You need to travel the same minimum distance from a hypothetical "Point A" to "Point B" no matter what, but the number of minutes could be inflated easily. Economic factors discourage inflating the number of miles. > There are so many ways to slice and dice this and come up with favourable results for self-driving systems that I can't but be dubious of any claim. I think it's fair to criticize the fact that they didn't adjust for geographic area with as much granularity as they should have (i.e. for highways vs city streets, or for excluded zones like airports). But other than that, their methodology seems solid to me.

necovek 2y ago
I understand that viewpoint from an insurance perspective, but not necessarily from my own safety perspective: I wonder how well Waymo would fare in circumstances I see as a driver. Eg. humans can control whether they drive under the influence or not (eg. if you don't take your car to a bar, you ain't coming back driving it). But most notably, aggregated statistics like these focus on comparisons similar to comparing averages vs better statistical representations (like average vs median or p75 or... eg. it's possible for one driver to cause 20 collisions, and in a set of 10 drivers with no other collisions, we'd be at 2 collisions per driver instead of 0 really). It's also not only about highway/city street miles, it's also about time of day: if Waymo is proportionally less on the streets and areas where there is more risky driving (eg. around bars and around midnight to 3am when there are also drunk pedestrians around) compared to human drivers, that would obviously skew the numbers in SDV's favour. Again, potentially they did since they acknowledge a separate study focusing on those, but later on they only talk about aggregated insurance claims. They also acknowledge they are not accounting for accident severity, not even by using the dollar amount: while they had no fatalities, it's obviously important to weigh accidents by their severity. With all this said, this does demonstrate current insurance liability of an average human driver compared to a Waymo SDV in Phoenix and SFO, but only once we have more comparable data should we make a bigger claim.
- hnaccount_rng 2y ago
  I get where you are coming from and you are correct. This is, up to a point, a PR piece of Waymo and we should take that into account. But they are pretty transparent about what they are doing. You just have to read the report. And for their metric, which rounds to "all distance travelled in the interesting ZIP codes", they are succeeding. What I'd take away from this is: If you give them a different metric, they can optimise for that too and they will end up being better than humans. That doesn't answer the questions: 1) Is it worth it? (I.e. what does it cost to achieve that) and 2) Is a Waymo better in any situation? I guess they don't have the answer to either yet. I'm pretty sure their cost-per-mile is still dropping like a rock (probably though not fast enough for their accountants) and they haven't reached all situations yet either. But I wouldn't bet on "because it's too hard", but rather "because we didn't get there yet" (and yes, they will have started with the easy part, as they should)
foobarqux 2y ago
> No, they didn't. Their methodology didn't adjust for things like highway vs city street miles, although it did adjust for city and state. Doesn't Waymo only drive in the city? And damage/injury is much greater on highways? If so that pretty much makes the study worthless.
- Reubend 2y ago
  AFAIK they drive on highways for testing, but they don't offer commercial rides on highway routes to customers yet. If that's still the case, I'd expect the majority of their miles to be on city streets. As I already pointed out, that's a big flaw in their methodology.

necovek 2y ago

I understand that viewpoint from an insurance perspective, but not necessarily from my own safety perspective: I wonder how well Waymo would fare in circumstances I see as a driver. Eg. humans can control whether they drive under the influence or not (eg. if you don't take your car to a bar, you ain't coming back driving it). But most notably, aggregated statistics like these focus on comparisons similar to comparing averages vs better statistical representations (like average vs median or p75 or... eg. it's possible for one driver to cause 20 collisions, and in a set of 10 drivers with no other collisions, we'd be at 2 collisions per driver instead of 0 really). It's also not only about highway/city street miles, it's also about time of day: if Waymo is proportionally less on the streets and areas where there is more risky driving (eg. around bars and around midnight to 3am when there are also drunk pedestrians around) compared to human drivers, that would obviously skew the numbers in SDV's favour. Again, potentially they did since they acknowledge a separate study focusing on those, but later on they only talk about aggregated insurance claims. They also acknowledge they are not accounting for accident severity, not even by using the dollar amount: while they had no fatalities, it's obviously important to weigh accidents by their severity. With all this said, this does demonstrate current insurance liability of an average human driver compared to a Waymo SDV in Phoenix and SFO, but only once we have more comparable data should we make a bigger claim.

hnaccount_rng 2y ago
I get where you are coming from and you are correct. This is, up to a point, a PR piece of Waymo and we should take that into account. But they are pretty transparent about what they are doing. You just have to read the report. And for their metric, which rounds to "all distance travelled in the interesting ZIP codes", they are succeeding. What I'd take away from this is: If you give them a different metric, they can optimise for that too and they will end up being better than humans. That doesn't answer the questions: 1) Is it worth it? (I.e. what does it cost to achieve that) and 2) Is a Waymo better in any situation? I guess they don't have the answer to either yet. I'm pretty sure their cost-per-mile is still dropping like a rock (probably though not fast enough for their accountants) and they haven't reached all situations yet either. But I wouldn't bet on "because it's too hard", but rather "because we didn't get there yet" (and yes, they will have started with the easy part, as they should)

foobarqux 2y ago

> No, they didn't. Their methodology didn't adjust for things like highway vs city street miles, although it did adjust for city and state. Doesn't Waymo only drive in the city? And damage/injury is much greater on highways? If so that pretty much makes the study worthless.

Reubend 2y ago
AFAIK they drive on highways for testing, but they don't offer commercial rides on highway routes to customers yet. If that's still the case, I'd expect the majority of their miles to be on city streets. As I already pointed out, that's a big flaw in their methodology.