6 ms·

I asked a question on Twitter a few weeks/months ago, and didn't receive a very good answer: At what point do we have enough data?

by dpcx 7y ago

semi-extrinsic 7y ago
To be honest: probably at around 50 billion miles driven with a single version of the autonomous system without any fatalities. That would be a milestone where you could say without a doubt "we've made a leap change to car safety". The gold standard for other safety critical systems like medical devices, commercial aviation etc. is less than one fatality per billion operating hours. 50 mph avg gets you 50 billion miles. A vision for car safety that doesn't approach this level isn't much to write home about, it would in fact be difficult to distinguish from just waiting 15 years to let the statistics incorporate the safety systems that already exist today.
- shafyy 7y ago
  According to Wikipedia there were 1.16 fatalities per 100 vehicle miles traveled in 2017. Why do you say autonomous cars need to be over 50x safer to be better than human drivers? Source: https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in_U.S._by_year https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in... Edit: Forgot to add year.
  - semi-extrinsic 7y ago
    Because that number is averaged over all types of vehicles, in all types of weather conditions. It includes people crashing in their '95 Honda Civic. It includes people forgetting to look over their shoulder when switching lanes. Crash survivability and "non-smart" safety features like blind spot warning has increased significantly the past ten-fifteen years. Just waiting until most of the old cars on the road today are replaced with newer ones will yield a 5x to 10x improvement, if we keep the human drivers, assuming no new tech at all. For instance the UK is already below 0.5 fatalities per 100 million vehicle miles. Car demographics matter a lot. So if autonomous is going to give us a significant improvement, it has to be something like 50x better than the average for humans today.
    - shafyy 7y ago
      Interesting. Do you have statistics for more modern cars only that are comparable to a Tesla (without the Autopilot features obviously)? PS: I meant 100 million miles in my comment above as you probably assumed :-)
      - semi-extrinsic 7y ago
        It's kinda hard to get those statistics, as new cars havent been driven that long, yet! But you can probably get a good estimate from extrapolating the red curve in this plot using a power law: https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in_U.S._by_year#/media/File%3AUS_traffic_deaths_per_VMT%2C_VMT%2C_per_capita%2C_and_total_annual_deaths.png https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in... Also you can look at the safest countries worldwide, compared to the US, to get a feel for what's achievable with today's tech. Looks like Norway is the safest, with 4x better safety statistics than the US. That's probably a combination of better drivers education and safer vehicles, so not straight forward to compare. So really, you would need something like a 50x improvement over current US rates for it to be a notable achievement. https://en.wikipedia.org/wiki/List_of_countries_by_traffic-related_death_rate https://en.wikipedia.org/wiki/List_of_countries_by_traffic-r...

I asked a question on Twitter a few weeks/months ago, and didn't receive a very good answer: At what point do we have enough data?

by dpcx 7y ago

I asked a question on Twitter a few weeks/months ago, and didn't receive a very good answer: At what point do we have enough data?

semi-extrinsic 7y ago

To be honest: probably at around 50 billion miles driven with a single version of the autonomous system without any fatalities. That would be a milestone where you could say without a doubt "we've made a leap change to car safety". The gold standard for other safety critical systems like medical devices, commercial aviation etc. is less than one fatality per billion operating hours. 50 mph avg gets you 50 billion miles. A vision for car safety that doesn't approach this level isn't much to write home about, it would in fact be difficult to distinguish from just waiting 15 years to let the statistics incorporate the safety systems that already exist today.

shafyy 7y ago
According to Wikipedia there were 1.16 fatalities per 100 vehicle miles traveled in 2017. Why do you say autonomous cars need to be over 50x safer to be better than human drivers? Source: https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in_U.S._by_year https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in... Edit: Forgot to add year.
- semi-extrinsic 7y ago
  Because that number is averaged over all types of vehicles, in all types of weather conditions. It includes people crashing in their '95 Honda Civic. It includes people forgetting to look over their shoulder when switching lanes. Crash survivability and "non-smart" safety features like blind spot warning has increased significantly the past ten-fifteen years. Just waiting until most of the old cars on the road today are replaced with newer ones will yield a 5x to 10x improvement, if we keep the human drivers, assuming no new tech at all. For instance the UK is already below 0.5 fatalities per 100 million vehicle miles. Car demographics matter a lot. So if autonomous is going to give us a significant improvement, it has to be something like 50x better than the average for humans today.
  - shafyy 7y ago
    Interesting. Do you have statistics for more modern cars only that are comparable to a Tesla (without the Autopilot features obviously)? PS: I meant 100 million miles in my comment above as you probably assumed :-)
    - semi-extrinsic 7y ago
      It's kinda hard to get those statistics, as new cars havent been driven that long, yet! But you can probably get a good estimate from extrapolating the red curve in this plot using a power law: https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in_U.S._by_year#/media/File%3AUS_traffic_deaths_per_VMT%2C_VMT%2C_per_capita%2C_and_total_annual_deaths.png https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in... Also you can look at the safest countries worldwide, compared to the US, to get a feel for what's achievable with today's tech. Looks like Norway is the safest, with 4x better safety statistics than the US. That's probably a combination of better drivers education and safer vehicles, so not straight forward to compare. So really, you would need something like a 50x improvement over current US rates for it to be a notable achievement. https://en.wikipedia.org/wiki/List_of_countries_by_traffic-related_death_rate https://en.wikipedia.org/wiki/List_of_countries_by_traffic-r...