7 ms·

Amen. Uber's fatal accident rate is now 50x that of sober human driver's. At their current rate of driving, they can't have a _lower_ rate until 2028! Inexcusab

Amen. Uber's fatal accident rate is now 50x that of sober human driver's. At their current rate of driving, they can't have a _lower_ rate until 2028! Inexcusable failure of their technology to prevent a human death in the most basic collision avoidance scenario.

setra 9y ago
A fatal accident at 50x the rate of a sober human driver with a study size where N = 1.
- coinjobber 9y ago
  No. 3 million miles of observation. You can get a pretty exact and conservative estimate with a bayesian poisson process model. I don't have the time to run the numbers right now, but my guess is the posterior estimate that Uber's fatal accident rate is higher than a human's is >90%, even if taking the human accident rate as a starting prior.
  - jfoutz 9y ago
    I thought Uber had to have a human take over every 13 miles. It’s more like 10 miles of observation 300,000 times. Or rather an attentive human can be 50x better than average.
  - btrettel 9y ago
    I'd be very interested in seeing the math if you have the time later.
    - gpm 9y ago
      95% - erring on assuming Uber has driven more miles than they probably have. https://news.ycombinator.com/item?id=16621118 https://news.ycombinator.com/item?id=16621118
      - comex 9y ago
        Hmm; if I understand correctly, in that link you show that if Uber’s AI has the same risk of killing people as a human driver, then the prior probability of an accident occurring when it did or earlier was 5%. That’s significant, but it’s not the same measure as the probability that the AI has a higher risk (which would require a prior distribution).
- tedsanders 9y ago
  It's a reasonable gut feeling to not generalize from n=1, but the numerical evidence - with either a Bayesian or frequentist approach - is actually quite strong and statistically significant. Math here: https://news.ycombinator.com/item?id=16655081 https://news.ycombinator.com/item?id=16655081
  - YeGoblynQueenne 9y ago
    That's not right. You're setting your expectation for N = 100m miles, then updating it for N = 3 million miles? That's like saying: "I rolled this red d20 twenty times before I rolled a 1, whereas I rolled a 1 the first time on this blue d20, so the red d20 is obviously better and I'm rolling all my saves on it". Or, I don't know- "I rolled three 1s on this d20 in twenty rolls so it's obviously not a fair d20".
    - tedsanders 9y ago
      Can you clarify? What do you believe to be wrong and why? If you have a strong prior the dice are equivalent, then yes, the rolls shouldn't change your mind. If you have a prior that the dice are weighted in an unknown way, then yes, the rolls really should change your mind.
- pgreenwood 9y ago
  I'll have a go at seeing what we can conclude from the data. Others, check my thinking please. Now we have 1 death in 3m miles for Uber, versus 1.18 deaths in 100m miles for sober drivers. The expected rate for 100m miles for Uber is 33.333... But how confident can we be? To answer that let's compute a poisson confidence interval around that rate, as in https://stats.stackexchange.com/questions/10926/how-to-calculate-confidence-interval-for-count-data-in-r https://stats.stackexchange.com/questions/10926/how-to-calcu.... Let's see what a 95% confidence interval for 1 death in 3m miles looks like: > poisson.test(1,conf.level = 0.95)$conf.int [1] 0.02531781 5.57164339 attr(,"conf.level") [1] 0.95 Multiply that by 33.333 to convert to deaths per 100m miles: > 33.333333*0.02531781 [1] 0.843927 > 33.333333*5.57164339 [1] 185.7214 So 95% confidence that the rate per 100m miles is from 0.84 to 185.72. That's pretty wide! And since the lower bound crosses 1.18, the difference is not significant at the .05 level (if we must make that particular comparison). However, let's look at 90% CI: > poisson.test(1,conf.level = 0.9)$conf.int [1] 0.05129329 4.74386452 attr(,"conf.level") [1] 0.9 Which gives a CI of 1.71 to 158.13. So with 90% confidence we can say Uber is less safe than sober drivers. Ok. Now let's look at 93% CI: > poisson.test(1,conf.level = 0.93)$conf.int [1] 0.03562718 5.17251332 attr(,"conf.level") [1] 0.93 That gives a CI of 1.188 to 172.417. The lower bound being just a bit worse than sober drivers. So we can conclude with 93% certainty from this data that Uber is less safe than sober drivers. Probably a LOT less safe. Although the CI is really wide, this is shocking data for Uber, in my opinion.
  - jakobegger 9y ago
    Erm... I don’t think statistics work like this. You can‘t go and pick a confidence level that „confirms“ your desired outcome. People with more knowledge about statistics than me might be able to explain why.
    - Slartie 9y ago
      Statistics works exactly like this. What doesn't work is saying "Okay, we have one death in 3 million miles, that extrapolates to 33 deaths in 100 million miles", because it implies a silent addition of "with nearly 100% certainty", which is the part that's wrong here. But the poster did something different. He took it one level further and attempted to calculate this confidence number for different spans in which the actual "deaths per 100 million miles" number of Uber's current cars would fall into, given an ideal world (from a data perspective) in which they would have driven an infinite amount of miles. But he actually did it the other way round - he modified the confidence variable and calculated the spans, and then he adjusted the confidence until he arrived at a span that would put Uber's cars just on par with human driving in the best case. The fact that a fatal incident happens that early (at 3 million, and not closer or past the 86 million that a statistical human drives on average until a fatal incident occurs) does not allow us to extrapolate a sound number per 100 million miles, but it tells us something about the probability by which the actual number of fatalities by 100 million miles that we'd get if Uber continued testing just like it did and racked up enough miles (and killed people) for a statistically sound calculation will fall into different margins. Sure, Uber could have been just very, very unlucky - but that's pretty unlikely, and the unlikeliness of Uber's bad luck (and conversely the likeliness of the fact that Uber's tech is just systematically deadly) is precisely what can be calculated with this single incident.
savanaly 9y ago
What is it compared to human drivers in general? That seems to be the fair comparison as "computer will never be drunk/tired/distracted" is usually cited as one of the benefits of computer-driven vehicles.
- galdosdi 9y ago
  Haha, as I joked in another article comment thread, their current crash rate ironically means a somewhat drunk but otherwise defensive driver is probably much safer! What a world.
- caf 9y ago
  If you want to be legally allowed on the road, I think your benchmark has to be other drivers that are legally allowed on the road.
agildehaus 9y ago
Before the accident they were infinitely better than a sober human driver. Understand statistics before employing them please. We don't have enough data and a single data point doesn't change that.
- deleted 9y ago
  [deleted]
- deleted 9y ago
  [deleted]
- raverbashing 9y ago
  I wonder if this type of events are modeled by Poisson processes and measured by MTB(A?)
- random314 9y ago
  > Understand statistics before employing them please Do you understand statistics? https://news.ycombinator.com/item?id=16685929 https://news.ycombinator.com/item?id=16685929
- simion314 9y ago
  It is not 1 data pint though, say you shot at a target 100 times and hit only 1 time , is this 1 data point only and I can't make any conclusion? Similar if you drive 1 million km and kill 1 person, if I drive 10 km and kill 1 person is still 1 data point and I can make no conclusion? I think it would have been 1 data point if this was the first km a Uber self driving car has driven.
  - YeGoblynQueenne 9y ago
    >> Similar if you drive 1 million km and kill 1 person, if I drive 10 km and kill 1 person is still 1 data point and I can make no conclusion? Yep. Because I still have another 9 m km to go before I've driven as long as you have and there is no way to know whether I'm going to kill another 9 people, or 0 more, until I've actually driven them all.

setra 9y ago

A fatal accident at 50x the rate of a sober human driver with a study size where N = 1.

coinjobber 9y ago
No. 3 million miles of observation. You can get a pretty exact and conservative estimate with a bayesian poisson process model. I don't have the time to run the numbers right now, but my guess is the posterior estimate that Uber's fatal accident rate is higher than a human's is >90%, even if taking the human accident rate as a starting prior.
- jfoutz 9y ago
  I thought Uber had to have a human take over every 13 miles. It’s more like 10 miles of observation 300,000 times. Or rather an attentive human can be 50x better than average.
- btrettel 9y ago
  I'd be very interested in seeing the math if you have the time later.
  - gpm 9y ago
    95% - erring on assuming Uber has driven more miles than they probably have. https://news.ycombinator.com/item?id=16621118 https://news.ycombinator.com/item?id=16621118
    - comex 9y ago
      Hmm; if I understand correctly, in that link you show that if Uber’s AI has the same risk of killing people as a human driver, then the prior probability of an accident occurring when it did or earlier was 5%. That’s significant, but it’s not the same measure as the probability that the AI has a higher risk (which would require a prior distribution).
tedsanders 9y ago
It's a reasonable gut feeling to not generalize from n=1, but the numerical evidence - with either a Bayesian or frequentist approach - is actually quite strong and statistically significant. Math here: https://news.ycombinator.com/item?id=16655081 https://news.ycombinator.com/item?id=16655081
- YeGoblynQueenne 9y ago
  That's not right. You're setting your expectation for N = 100m miles, then updating it for N = 3 million miles? That's like saying: "I rolled this red d20 twenty times before I rolled a 1, whereas I rolled a 1 the first time on this blue d20, so the red d20 is obviously better and I'm rolling all my saves on it". Or, I don't know- "I rolled three 1s on this d20 in twenty rolls so it's obviously not a fair d20".
  - tedsanders 9y ago
    Can you clarify? What do you believe to be wrong and why? If you have a strong prior the dice are equivalent, then yes, the rolls shouldn't change your mind. If you have a prior that the dice are weighted in an unknown way, then yes, the rolls really should change your mind.
pgreenwood 9y ago
I'll have a go at seeing what we can conclude from the data. Others, check my thinking please. Now we have 1 death in 3m miles for Uber, versus 1.18 deaths in 100m miles for sober drivers. The expected rate for 100m miles for Uber is 33.333... But how confident can we be? To answer that let's compute a poisson confidence interval around that rate, as in https://stats.stackexchange.com/questions/10926/how-to-calculate-confidence-interval-for-count-data-in-r https://stats.stackexchange.com/questions/10926/how-to-calcu.... Let's see what a 95% confidence interval for 1 death in 3m miles looks like: > poisson.test(1,conf.level = 0.95)$conf.int [1] 0.02531781 5.57164339 attr(,"conf.level") [1] 0.95 Multiply that by 33.333 to convert to deaths per 100m miles: > 33.333333*0.02531781 [1] 0.843927 > 33.333333*5.57164339 [1] 185.7214 So 95% confidence that the rate per 100m miles is from 0.84 to 185.72. That's pretty wide! And since the lower bound crosses 1.18, the difference is not significant at the .05 level (if we must make that particular comparison). However, let's look at 90% CI: > poisson.test(1,conf.level = 0.9)$conf.int [1] 0.05129329 4.74386452 attr(,"conf.level") [1] 0.9 Which gives a CI of 1.71 to 158.13. So with 90% confidence we can say Uber is less safe than sober drivers. Ok. Now let's look at 93% CI: > poisson.test(1,conf.level = 0.93)$conf.int [1] 0.03562718 5.17251332 attr(,"conf.level") [1] 0.93 That gives a CI of 1.188 to 172.417. The lower bound being just a bit worse than sober drivers. So we can conclude with 93% certainty from this data that Uber is less safe than sober drivers. Probably a LOT less safe. Although the CI is really wide, this is shocking data for Uber, in my opinion.
- jakobegger 9y ago
  Erm... I don’t think statistics work like this. You can‘t go and pick a confidence level that „confirms“ your desired outcome. People with more knowledge about statistics than me might be able to explain why.
  - Slartie 9y ago
    Statistics works exactly like this. What doesn't work is saying "Okay, we have one death in 3 million miles, that extrapolates to 33 deaths in 100 million miles", because it implies a silent addition of "with nearly 100% certainty", which is the part that's wrong here. But the poster did something different. He took it one level further and attempted to calculate this confidence number for different spans in which the actual "deaths per 100 million miles" number of Uber's current cars would fall into, given an ideal world (from a data perspective) in which they would have driven an infinite amount of miles. But he actually did it the other way round - he modified the confidence variable and calculated the spans, and then he adjusted the confidence until he arrived at a span that would put Uber's cars just on par with human driving in the best case. The fact that a fatal incident happens that early (at 3 million, and not closer or past the 86 million that a statistical human drives on average until a fatal incident occurs) does not allow us to extrapolate a sound number per 100 million miles, but it tells us something about the probability by which the actual number of fatalities by 100 million miles that we'd get if Uber continued testing just like it did and racked up enough miles (and killed people) for a statistically sound calculation will fall into different margins. Sure, Uber could have been just very, very unlucky - but that's pretty unlikely, and the unlikeliness of Uber's bad luck (and conversely the likeliness of the fact that Uber's tech is just systematically deadly) is precisely what can be calculated with this single incident.

savanaly 9y ago

What is it compared to human drivers in general? That seems to be the fair comparison as "computer will never be drunk/tired/distracted" is usually cited as one of the benefits of computer-driven vehicles.

galdosdi 9y ago
Haha, as I joked in another article comment thread, their current crash rate ironically means a somewhat drunk but otherwise defensive driver is probably much safer! What a world.
caf 9y ago
If you want to be legally allowed on the road, I think your benchmark has to be other drivers that are legally allowed on the road.

agildehaus 9y ago

Before the accident they were infinitely better than a sober human driver. Understand statistics before employing them please. We don't have enough data and a single data point doesn't change that.

deleted 9y ago
[deleted]
deleted 9y ago
[deleted]
raverbashing 9y ago
I wonder if this type of events are modeled by Poisson processes and measured by MTB(A?)
random314 9y ago
> Understand statistics before employing them please Do you understand statistics? https://news.ycombinator.com/item?id=16685929 https://news.ycombinator.com/item?id=16685929
simion314 9y ago
It is not 1 data pint though, say you shot at a target 100 times and hit only 1 time , is this 1 data point only and I can't make any conclusion? Similar if you drive 1 million km and kill 1 person, if I drive 10 km and kill 1 person is still 1 data point and I can make no conclusion? I think it would have been 1 data point if this was the first km a Uber self driving car has driven.
- YeGoblynQueenne 9y ago
  >> Similar if you drive 1 million km and kill 1 person, if I drive 10 km and kill 1 person is still 1 data point and I can make no conclusion? Yep. Because I still have another 9 m km to go before I've driven as long as you have and there is no way to know whether I'm going to kill another 9 people, or 0 more, until I've actually driven them all.