4 ms·

According to their latest Californian DMV filing, Waymo's disengagement rate was 0.076 per 1,000 miles. This means for every disengagement, they've driven for

by dannyw 7y ago

According to their latest Californian DMV filing, Waymo's disengagement rate was 0.076 per 1,000 miles. This means for every disengagement, they've driven for 13,158 miles on average. That's ~1000~ EDIT: 4, thank you; orders of magnitudes from your "5km". Could a Mercedes in the 1990s drive itself for 13k miles without human intervention?

baybal2 7y ago
This is a debated number, with people telling that they don't count anything, but actual "stall" when the system stands still and can't do anything, as disengagement. And accounts of stalled Waymos in the middle of the road are too frequent to believe them
- jeen02 7y ago
  > And accounts of stalled Waymos in the middle of the road are too frequent to believe them You gonna provide a source for that?
- azernik 7y ago
  Disengagement is whenever the human test driver has to intervene. The car stalling in the middle of the road would only be counted as a disengagement if it doesn't recover on its own.
graphpapa 7y ago
13000m ~> 20000km Order of 200000: 10^4 Order of 5: 1^0 Conclusion: difference in order of magnitudes = 4 (not 1000)
ohazi 7y ago
> This means for every disengagement, they've driven for 13,158 miles on average. That's 1000 orders of magnitudes from your "5km". That's four orders of magnitude, not a thousand.
- dannyw 7y ago
  Thank you for the correction :)
TeMPOraL 7y ago
I don't trust that this number is proving what you're saying it does. It's a meta point, but I found myself trusting such "data" less and less. "Do you have data?" or "Do you have a peer-reviewed paper about it?" is an Internet meme at this point. Bringing some numbers or DOIs is table stakes now, in anything but dumbest of discussions. These values - "0.076 per 1000 miles", "13158 miles on average" - have a history behind them. They're pulled out of a set of test drives. But how did the drives look? Was it a collection of similar drives, or a bunch of longer drives with significant disengagement plus a lot of short and easy drives to lower the average? To understand the true meaning of the numbers, you'd need to see (among other things) the distribution of distances covered by test drives. I wish we had a way to require and embed context with the data, inline with the data. So that when I see "0.0076 per 1000 miles", I know it's: SELECT 1000 * (SELECT sum(disengagements) FROM drives) / (SELECT sum(distance) FROM drives) And so that I could click around and turn it into e.g. a probability distribution of: SELECT distance FROM drives Or view a cumulative distribution of disengagements per distance: SELECT t1.distance, SUM(t2.disengagements) FROM drives t1 INNER JOIN drives t2 ON t1.distance >= t2.distance GROUP BY t1.distance ORDER BY t1.distance Now the problem usually is, this data is often not available - people release aggregate summaries instead of full data sets (whether that's the case with Waymo, I don't know - I didn't check, because I don't care about this particular case). But it should be made available, because point summaries of distributions are the easiest way to derive bullshit conclusions (and/or intentionally mislead people). And conversely, I don't trust point summaries unless I either a) review the underlying data (or at least know the shape of its distribution), or b) can trust that the person giving me the summary reviewed the underlying data (or knows the shape of its distribution) and isn't trying to mislead. So there are two problems here: getting people to release more data, and creating tech infrastructure so that this data can be explored inline (preferably with nice UX that doesn't require user to type in SQL). </end-of-meta-rant>
kohtatsu 7y ago
FWIW, 1,000 orders of magnitude is an order of magnitude larger than a googol. Also where these miles happen, and how long they take relative to a human driver (1.1x?) matter quite a bit. edit: I've never shared a road with a self-driving car but I can't imagine I'd like it. I'm extremely patient with fellow meatbags (I work a lot of customer service, and enjoy it), but I'm not at all patient with software thinking it's smarter than it is. I also think people tend to underestimate how much "car-body language" is involved with driving as well, but I suppose that's the primary thing self-driving cars are learning. Self-driving cars can also burgeon a surveillance state depending on how the fleet is implemented, particularly around what will be required for client-side redaction (pedestrians, license plates, windows etc). Keeping raw data locally for a period of time is fine in case of crash/malfunction, even backing it up encrypted with ephemeral keys can be well and good. But unfettered collection and access to unredacted data—perhaps something Google is drooling over—needs to be stopped cold.
- zingermc 7y ago
  To be fully pedantic, 1000 orders of magnitude (10^1000) is really 900 orders of magnitude larger than a googol. 10^1000 / 10^100 = 10^900
  - kohtatsu 7y ago
    Woah, cool thank you.
stefan_ 7y ago
There are endless ways to game that number. The biggest systematic bias of all must be simply California weather.
- tialaramex 7y ago
  That's a pretty clever technology though right? Because ordinarily you'd expect Waymo cars driving in say Phoenix, Arizona to have er, Arizona weather? More seriously, once you switch to "Actually I think in a snow storm this car isn't better after all" we're back to God of the Gaps style arguments which you can choose to lose incrementally if you want but it just looks kind of sad. It's definitely true that human drivers will set out in dangerous conditions and sometimes arrive safely. If, as Waymo promotes, the idea is self-driving cars are safer they just won't go in those conditions because it's unsafe.

by dannyw 7y ago

baybal2 7y ago

This is a debated number, with people telling that they don't count anything, but actual "stall" when the system stands still and can't do anything, as disengagement. And accounts of stalled Waymos in the middle of the road are too frequent to believe them

jeen02 7y ago
> And accounts of stalled Waymos in the middle of the road are too frequent to believe them You gonna provide a source for that?
azernik 7y ago
Disengagement is whenever the human test driver has to intervene. The car stalling in the middle of the road would only be counted as a disengagement if it doesn't recover on its own.

graphpapa 7y ago

13000m ~> 20000km Order of 200000: 10^4 Order of 5: 1^0 Conclusion: difference in order of magnitudes = 4 (not 1000)

ohazi 7y ago

> This means for every disengagement, they've driven for 13,158 miles on average. That's 1000 orders of magnitudes from your "5km". That's four orders of magnitude, not a thousand.

dannyw 7y ago
Thank you for the correction :)

TeMPOraL 7y ago

I don't trust that this number is proving what you're saying it does. It's a meta point, but I found myself trusting such "data" less and less. "Do you have data?" or "Do you have a peer-reviewed paper about it?" is an Internet meme at this point. Bringing some numbers or DOIs is table stakes now, in anything but dumbest of discussions. These values - "0.076 per 1000 miles", "13158 miles on average" - have a history behind them. They're pulled out of a set of test drives. But how did the drives look? Was it a collection of similar drives, or a bunch of longer drives with significant disengagement plus a lot of short and easy drives to lower the average? To understand the true meaning of the numbers, you'd need to see (among other things) the distribution of distances covered by test drives. I wish we had a way to require and embed context with the data, inline with the data. So that when I see "0.0076 per 1000 miles", I know it's: SELECT 1000 * (SELECT sum(disengagements) FROM drives) / (SELECT sum(distance) FROM drives) And so that I could click around and turn it into e.g. a probability distribution of: SELECT distance FROM drives Or view a cumulative distribution of disengagements per distance: SELECT t1.distance, SUM(t2.disengagements) FROM drives t1 INNER JOIN drives t2 ON t1.distance >= t2.distance GROUP BY t1.distance ORDER BY t1.distance Now the problem usually is, this data is often not available - people release aggregate summaries instead of full data sets (whether that's the case with Waymo, I don't know - I didn't check, because I don't care about this particular case). But it should be made available, because point summaries of distributions are the easiest way to derive bullshit conclusions (and/or intentionally mislead people). And conversely, I don't trust point summaries unless I either a) review the underlying data (or at least know the shape of its distribution), or b) can trust that the person giving me the summary reviewed the underlying data (or knows the shape of its distribution) and isn't trying to mislead. So there are two problems here: getting people to release more data, and creating tech infrastructure so that this data can be explored inline (preferably with nice UX that doesn't require user to type in SQL). </end-of-meta-rant>

kohtatsu 7y ago

FWIW, 1,000 orders of magnitude is an order of magnitude larger than a googol. Also where these miles happen, and how long they take relative to a human driver (1.1x?) matter quite a bit. edit: I've never shared a road with a self-driving car but I can't imagine I'd like it. I'm extremely patient with fellow meatbags (I work a lot of customer service, and enjoy it), but I'm not at all patient with software thinking it's smarter than it is. I also think people tend to underestimate how much "car-body language" is involved with driving as well, but I suppose that's the primary thing self-driving cars are learning. Self-driving cars can also burgeon a surveillance state depending on how the fleet is implemented, particularly around what will be required for client-side redaction (pedestrians, license plates, windows etc). Keeping raw data locally for a period of time is fine in case of crash/malfunction, even backing it up encrypted with ephemeral keys can be well and good. But unfettered collection and access to unredacted data—perhaps something Google is drooling over—needs to be stopped cold.

zingermc 7y ago
To be fully pedantic, 1000 orders of magnitude (10^1000) is really 900 orders of magnitude larger than a googol. 10^1000 / 10^100 = 10^900
- kohtatsu 7y ago
  Woah, cool thank you.

stefan_ 7y ago

There are endless ways to game that number. The biggest systematic bias of all must be simply California weather.

tialaramex 7y ago
That's a pretty clever technology though right? Because ordinarily you'd expect Waymo cars driving in say Phoenix, Arizona to have er, Arizona weather? More seriously, once you switch to "Actually I think in a snow storm this car isn't better after all" we're back to God of the Gaps style arguments which you can choose to lose incrementally if you want but it just looks kind of sad. It's definitely true that human drivers will set out in dangerous conditions and sometimes arrive safely. If, as Waymo promotes, the idea is self-driving cars are safer they just won't go in those conditions because it's unsafe.