6 ms·

At the end of 2016 all companies testing autonomous vehicles in California filed their disengagement reports. Waymo reported 1 unplanned disengagement every 500

At the end of 2016 all companies testing autonomous vehicles in California filed their disengagement reports. Waymo reported 1 unplanned disengagement every 5000 miles, and GM reported 1 every 180. Nobody else was even in the running. These metrics are crude, but it's currently the only benchmarking system we've got, and we'll see in December where Cruise is really at when they report their 2017 numbers. Cruise does distinguish itself by driving mostly on busy downtown SF streets, a challenging environment, and they've released impressive videos showing off their capabilities. A few weeks ago Kyle Vogt claimed Cruise would surpass Waymo 'in a few months', whether they do or not remains to be seen, but they are making sustained progress and they have the full force of General Motors behind them. Unlike Waymo, Tesla's Autopilot program, and Uber's autonomous driving program, Cruise's operation has been drama free, without any lawsuits, leadership blowouts, or departures of key people. This implies they've got a solid team under good leadership, and I think it matters a lot with complex software projects like this.

TwoFactor 9y ago
There was actually a dispute about a potential Cruise cofounder getting compensation for equity at the acquisition. They recently settled the lawsuit out of court [1]. I agree they have good leadership, but they're not without any lawsuits. [1] http://www.businessinsider.com/car-startup-cruise-settles-legal-case-involving-cofounder-2016-5 http://www.businessinsider.com/car-startup-cruise-settles-le...
biggc 9y ago
Is disengagement per X miles really a good metric. It doesn't reflect the difference between driving in different environments, i.e dense urban areas, suburban streets, and open highways.
- Fricken 9y ago
  Yes, it's a crude metric. The other thing is disengagements are self reported, and an unplanned disengagement doesn't necessarily mean a crash was narrowly avoided, it may be an overly cautious test driver, or the result of some non-safety critical intervention. After the core software problems have been solved, the key challenge shifts to discovering edge cases, which become progressively more marginal the further advanced the software becomes, and the only way to find these edge cases is through 1000s of miles of driving waiting for unexpected situations to show up.
Retric 9y ago
Waymo (Google) is not that far from level IV. Average U.S. driver has one accident roughly every 165,000 miles and disengagements is not 1:1 with crashing. I suspect drunk drivers would probably be safer handing over control which could make a huge difference. Most interesting is the Disengagements by location. (page 9) https://www.dmv.ca.gov/portal/wcm/connect/946b3502-c959-4e3b-b119-91319c27788f/GoogleAutoWaymo_disengage_report_2016.pdf?MOD=AJPERES https://www.dmv.ca.gov/portal/wcm/connect/946b3502-c959-4e3b... Interstate 0 Freeway 0 Highway 12 Street 112 Which might be good or bad depending on the relative amount of driving in each situation. PS: Disengagements per 1,000 miles also dropped from 0.8 (2015) to 0.2 (2016) which suggests very good things. PPS: Been getting a lot of up and down votes I wander what people agree / disagree with?
- aetherson 9y ago
  I don't know why you're getting downvotes. Does anyone know off-hand, how much testing have autonomous vehicles had at night (as in, full dark/need to use headlights).
  - the_mar 9y ago
    Cruise has been testing cars 24 hours a day, 7 days a week for quite some time. Here's a few hours of fully autonomous night driving: https://www.youtube.com/watch?v=6tA_VvHP0-s https://www.youtube.com/watch?v=6tA_VvHP0-s
- abalone 9y ago
  Sounds like you're assuming level 4 merely means parity or better with human error. Lots of problems with that. For instance if it's Google's negligent bug that kills the family of 4 instead of a driver, Google gets sued. Google has much deeper pockets than the driver who would typically declare bankruptcy, effectively capping the damages. No real cap for Google, every time. So that explodes the cost of autonomous vehicles. And then there's the matter of potential criminal liability in the case of gross negligence. So there's going to be a much higher bar for transitioning to level 4. Also, while there may be fewer of some types of accidents, there will be lawsuits for a potentially very large set of other types that never used to get litigated. For example today when a driver causes their own accident, nobody gets sued.
phkahler 9y ago
>> Cruise does distinguish itself by driving mostly on busy downtown SF streets, a challenging environment, and they've released impressive videos showing off their capabilities. I don't doubt that it's a challenging environment, but there are many, many different kinds of challenges that are not even present in that city.
- zzzzzzzza 9y ago
  True. But city driving is all they need to offer service in the most potentially profitable, geo fenced areas.
- spikels 9y ago
  Driving is SF is actually quite challenging. It is quite congested with cars as well as pedestrians and cyclists. Due to lax enforcement many traffic laws are routinely ignored particularly by cyclists, pedestrians and transit vehicles. On top of this there is lots of construction, poorly maintained roads and frequent traffic flow changes. Not sure what you are imagining but SF is free of winter driving conditions. Other than that it is as challenging as any city in the US, Europe or Japan.

TwoFactor 9y ago

There was actually a dispute about a potential Cruise cofounder getting compensation for equity at the acquisition. They recently settled the lawsuit out of court [1]. I agree they have good leadership, but they're not without any lawsuits. [1] http://www.businessinsider.com/car-startup-cruise-settles-legal-case-involving-cofounder-2016-5 http://www.businessinsider.com/car-startup-cruise-settles-le...

biggc 9y ago

Is disengagement per X miles really a good metric. It doesn't reflect the difference between driving in different environments, i.e dense urban areas, suburban streets, and open highways.

Fricken 9y ago
Yes, it's a crude metric. The other thing is disengagements are self reported, and an unplanned disengagement doesn't necessarily mean a crash was narrowly avoided, it may be an overly cautious test driver, or the result of some non-safety critical intervention. After the core software problems have been solved, the key challenge shifts to discovering edge cases, which become progressively more marginal the further advanced the software becomes, and the only way to find these edge cases is through 1000s of miles of driving waiting for unexpected situations to show up.

Retric 9y ago

Waymo (Google) is not that far from level IV. Average U.S. driver has one accident roughly every 165,000 miles and disengagements is not 1:1 with crashing. I suspect drunk drivers would probably be safer handing over control which could make a huge difference. Most interesting is the Disengagements by location. (page 9) https://www.dmv.ca.gov/portal/wcm/connect/946b3502-c959-4e3b-b119-91319c27788f/GoogleAutoWaymo_disengage_report_2016.pdf?MOD=AJPERES https://www.dmv.ca.gov/portal/wcm/connect/946b3502-c959-4e3b... Interstate 0 Freeway 0 Highway 12 Street 112 Which might be good or bad depending on the relative amount of driving in each situation. PS: Disengagements per 1,000 miles also dropped from 0.8 (2015) to 0.2 (2016) which suggests very good things. PPS: Been getting a lot of up and down votes I wander what people agree / disagree with?

aetherson 9y ago
I don't know why you're getting downvotes. Does anyone know off-hand, how much testing have autonomous vehicles had at night (as in, full dark/need to use headlights).
- the_mar 9y ago
  Cruise has been testing cars 24 hours a day, 7 days a week for quite some time. Here's a few hours of fully autonomous night driving: https://www.youtube.com/watch?v=6tA_VvHP0-s https://www.youtube.com/watch?v=6tA_VvHP0-s
abalone 9y ago
Sounds like you're assuming level 4 merely means parity or better with human error. Lots of problems with that. For instance if it's Google's negligent bug that kills the family of 4 instead of a driver, Google gets sued. Google has much deeper pockets than the driver who would typically declare bankruptcy, effectively capping the damages. No real cap for Google, every time. So that explodes the cost of autonomous vehicles. And then there's the matter of potential criminal liability in the case of gross negligence. So there's going to be a much higher bar for transitioning to level 4. Also, while there may be fewer of some types of accidents, there will be lawsuits for a potentially very large set of other types that never used to get litigated. For example today when a driver causes their own accident, nobody gets sued.

phkahler 9y ago

>> Cruise does distinguish itself by driving mostly on busy downtown SF streets, a challenging environment, and they've released impressive videos showing off their capabilities. I don't doubt that it's a challenging environment, but there are many, many different kinds of challenges that are not even present in that city.

zzzzzzzza 9y ago
True. But city driving is all they need to offer service in the most potentially profitable, geo fenced areas.
spikels 9y ago
Driving is SF is actually quite challenging. It is quite congested with cars as well as pedestrians and cyclists. Due to lax enforcement many traffic laws are routinely ignored particularly by cyclists, pedestrians and transit vehicles. On top of this there is lots of construction, poorly maintained roads and frequent traffic flow changes. Not sure what you are imagining but SF is free of winter driving conditions. Other than that it is as challenging as any city in the US, Europe or Japan.