11 ms·

"the number of fatalities that have occurred with autopilot active compared to the amount of miles driven with autopilot active suggest that autopilot is safer

"the number of fatalities that have occurred with autopilot active compared to the amount of miles driven with autopilot active suggest that autopilot is safer than a human driver." Obviously you see how this deduction is subject to dozens of biases stemming from the fact that people turn on autopilot when the road conditions are safe to begin with. One would have to really carefully filter this data to avoid these biases, otherwise such conclusions are absolutely bogus.

toomuchtodo 9y ago
The 40% reduction in accident rate was the result of the NTHSA investigation: https://techcrunch.com/2017/01/19/nhtsas-full-final-investigation-into-teslas-autopilot-shows-40-crash-rate-reduction/ https://techcrunch.com/2017/01/19/nhtsas-full-final-investig... "It’s essentially as good as result as Tesla can have hoped for from the U.S. traffic safety agency’s investigation, which took place over the last six months. Reuters reported earlier on Thursday that the investigation would not result in a recall of Tesla vehicles, but the full findings show that in fact, the federal regulatory body found plenty to praise while conducting its inquiry. The investigation does conclude with a minor admonition that Tesla could perhaps be more specific about its system limitations in its driver-assist features, but acknowledges that all required information is present and available to drivers and vehicle owners."
- felippee 9y ago
  I know this report. This does not deal with the bias problem however. Obviously it reduced the number of accidents in good road conditions where the autopilot works and is primarily used (much like auto-stop feature did and BLIS). Particularly after they introduced the driver monitoring (this report was conducted after that) - now in fact the driver using autopilot is likely more alert then one not using it. But that does not indicate anything about conditions where autopilot should not be used in which it is gravely dangerous. It is all about tail of the distribution, which why it is not so obvious.
  - semiel 9y ago
    Why does it matter if it's dangerous if used incorrectly, if the majority of users use it correctly, and the overall effect is that fatalities go down?
    - felippee 9y ago
      I don't think the argument here is whether technology features can improve safety. The core of the argument is (and always has been) over misleading marketing and naming (suggestive "autopilot"), which exactly encourages the misuse of the feature. When used correctly, with alert driver it can clearly be beneficial. But when misused e.g. by this poor young person who plowed into a truck, this feature could be deadly. This has a long way to go for true "autonomy" and until then such marketing should be avoided and examples of misuse (which youtube is full of) should be explicitly discouraged and if possible penalized. Tesla should be more explicit and pro-active about it. This is my only point really.
      - ScottBurson 9y ago
        40 years old is "young"? You must be as old as I am :-)
        felippee 9y ago
        I was thinking about the Chinese boy who hit the truck parked on the service lane, not the guy who decapitated himself in Florida. But either way, both were tragic.
    - usrusr 9y ago
      Put "don't crash into things" in the manual and all accidents are incorrect use by definition? That's why it matters.
    - yahna 9y ago
      uh, because if the effectiveness drops dramatically then you are literally going to be killing people.
- mcguire 9y ago
  That sounds like a comparison between autopilot miles and general driving miles. What would the comparison look like between autopilot miles and interstate highway miles? Edit: nope, I was wrong. 1.3 to 0.8 airbag deployment crashes per million miles for Tesla cars before and after autopilot release.
chc 9y ago
This seems like trying to argue that brakes are bad because, while people normally use them at times when they should stop, it's also possible to use your brakes at times when you shouldn't stop. The fact that something can be used used in a dumb manner is good to know so you can avoid doing so, but that doesn't negate the fact that cars are safer overall with it than without it.
- frgtpsswrdlame 9y ago
  Yes but if I asked you to prove brakes were safer you'd be able to do it pretty easily. The fact that no one has actually set out and done a study proving that autopilot is safer than human driving is disconcerting considering these vehicles are already being put on the road. Seeing people abandoning the precautionary principle so they can latch onto an over-marketed, over-promised Muskian view of the future is sad. And in five or ten years we're all going to be looking back wondering what made us so dumb to think that computers could actually drive cars.
  - chc 9y ago
    After the first fatality while on Autopilot, the NHTSA did a study on whether Autopilot was inherently safe and whether it had any defects that needed to be addressed. It found that Teslas with Autopilot were substantially safer than Teslas without. I'd like to see a better study (since that one was mainly aimed at answering the question of whether Teslas with Autopilot should be recalled), but someone has looked into it at least.
    - frgtpsswrdlame 9y ago
      That's not quite accurate, the NHTSA study was not designed, in any way, to compare human-driven miles and autopilot-driven miles for safety. The only thing it showed was that vehicles with TACC + Autosteer had less accidents than vehicles with just TACC. Whether TACC + Autosteer is safer than human drivers is up for debate.
      - 9y ago
dmix 9y ago
With the same logic if people use it when road conditions are safer then they are simply using it as intended. The design can help prevent people from using it when it's not appropriate or have mechanism to keep them alert and focused. That is all NTSB seems to be saying here... they aren't calling into question the idea of highway driving assist systems, which are (already) showing potential to significantly make highway driving safer - the time when people are driving at the highest speeds. No system will be perfect from preemptive testing either, it really does take real world mass adoption to detect some uncommon flaws unfortunately... Tesla is going to bear the brunt of this fact. You simply can't simulate every human (irrational) behaviour nor can you simulate every environment it will be used in.
URSpider94 9y ago
Note, the original NHTSA report compares accidents per mile traveled in Tesla vehicles with and without autopilot, and notes that vehicles WITH autopilot have 40% fewer accidents than those without. This seems like very clean data, as it doesn't actually consider how often autopilot is turned on, and it's over a fairly large sample of drivers, locales and miles driven. This is just correlation, not causality, but the effect is too big to be by chance.
- dwaltrip 9y ago
  The comment you are replying to is suggesting that the miles driven with autopilot may have been inherently safer miles, e.g. autopilot was more likely to be turned during the safer parts of a drive. If true, this would be an unfortunate example of selection bias.
  - URSpider94 9y ago
    You are missing the point. The report doesn't compare miles driven with and without autopilot. It compares ALL miles driven in cars with Autopilot function, whether it is turned on or not, vs. cars without Autopilot function. To boot, these are, for the most part, the same cars before and after, since Tesla shipped cars for almost a year with the Autopilot hardware built in but not activated, then activated it for all cars at virtually the same time via an OTA. Before that date, any miles clocked show up in the "no autopilot" column, the next day they show up in the "with Autopilot" column. To further flog an expired equine, the study showed a 40% reduction in airbag actuations per million miles driven across all miles driven in cars with vs. without Autopilot function, not comparing only miles where Autopilot was turned on. There are certainly possibilities for this data to be skewed or biased, but it's definitely not a comparison of safe Autopilot miles vs. unsafe manual miles.

toomuchtodo 9y ago

The 40% reduction in accident rate was the result of the NTHSA investigation: https://techcrunch.com/2017/01/19/nhtsas-full-final-investigation-into-teslas-autopilot-shows-40-crash-rate-reduction/ https://techcrunch.com/2017/01/19/nhtsas-full-final-investig... "It’s essentially as good as result as Tesla can have hoped for from the U.S. traffic safety agency’s investigation, which took place over the last six months. Reuters reported earlier on Thursday that the investigation would not result in a recall of Tesla vehicles, but the full findings show that in fact, the federal regulatory body found plenty to praise while conducting its inquiry. The investigation does conclude with a minor admonition that Tesla could perhaps be more specific about its system limitations in its driver-assist features, but acknowledges that all required information is present and available to drivers and vehicle owners."

felippee 9y ago
I know this report. This does not deal with the bias problem however. Obviously it reduced the number of accidents in good road conditions where the autopilot works and is primarily used (much like auto-stop feature did and BLIS). Particularly after they introduced the driver monitoring (this report was conducted after that) - now in fact the driver using autopilot is likely more alert then one not using it. But that does not indicate anything about conditions where autopilot should not be used in which it is gravely dangerous. It is all about tail of the distribution, which why it is not so obvious.
- semiel 9y ago
  Why does it matter if it's dangerous if used incorrectly, if the majority of users use it correctly, and the overall effect is that fatalities go down?
  - felippee 9y ago
    I don't think the argument here is whether technology features can improve safety. The core of the argument is (and always has been) over misleading marketing and naming (suggestive "autopilot"), which exactly encourages the misuse of the feature. When used correctly, with alert driver it can clearly be beneficial. But when misused e.g. by this poor young person who plowed into a truck, this feature could be deadly. This has a long way to go for true "autonomy" and until then such marketing should be avoided and examples of misuse (which youtube is full of) should be explicitly discouraged and if possible penalized. Tesla should be more explicit and pro-active about it. This is my only point really.
    - ScottBurson 9y ago
      40 years old is "young"? You must be as old as I am :-)
      - felippee 9y ago
        I was thinking about the Chinese boy who hit the truck parked on the service lane, not the guy who decapitated himself in Florida. But either way, both were tragic.
  - usrusr 9y ago
    Put "don't crash into things" in the manual and all accidents are incorrect use by definition? That's why it matters.
  - yahna 9y ago
    uh, because if the effectiveness drops dramatically then you are literally going to be killing people.
mcguire 9y ago
That sounds like a comparison between autopilot miles and general driving miles. What would the comparison look like between autopilot miles and interstate highway miles? Edit: nope, I was wrong. 1.3 to 0.8 airbag deployment crashes per million miles for Tesla cars before and after autopilot release.

chc 9y ago

This seems like trying to argue that brakes are bad because, while people normally use them at times when they should stop, it's also possible to use your brakes at times when you shouldn't stop. The fact that something can be used used in a dumb manner is good to know so you can avoid doing so, but that doesn't negate the fact that cars are safer overall with it than without it.

frgtpsswrdlame 9y ago
Yes but if I asked you to prove brakes were safer you'd be able to do it pretty easily. The fact that no one has actually set out and done a study proving that autopilot is safer than human driving is disconcerting considering these vehicles are already being put on the road. Seeing people abandoning the precautionary principle so they can latch onto an over-marketed, over-promised Muskian view of the future is sad. And in five or ten years we're all going to be looking back wondering what made us so dumb to think that computers could actually drive cars.
- chc 9y ago
  After the first fatality while on Autopilot, the NHTSA did a study on whether Autopilot was inherently safe and whether it had any defects that needed to be addressed. It found that Teslas with Autopilot were substantially safer than Teslas without. I'd like to see a better study (since that one was mainly aimed at answering the question of whether Teslas with Autopilot should be recalled), but someone has looked into it at least.
  - frgtpsswrdlame 9y ago
    That's not quite accurate, the NHTSA study was not designed, in any way, to compare human-driven miles and autopilot-driven miles for safety. The only thing it showed was that vehicles with TACC + Autosteer had less accidents than vehicles with just TACC. Whether TACC + Autosteer is safer than human drivers is up for debate.
    - 9y ago

dmix 9y ago

With the same logic if people use it when road conditions are safer then they are simply using it as intended. The design can help prevent people from using it when it's not appropriate or have mechanism to keep them alert and focused. That is all NTSB seems to be saying here... they aren't calling into question the idea of highway driving assist systems, which are (already) showing potential to significantly make highway driving safer - the time when people are driving at the highest speeds. No system will be perfect from preemptive testing either, it really does take real world mass adoption to detect some uncommon flaws unfortunately... Tesla is going to bear the brunt of this fact. You simply can't simulate every human (irrational) behaviour nor can you simulate every environment it will be used in.

URSpider94 9y ago

Note, the original NHTSA report compares accidents per mile traveled in Tesla vehicles with and without autopilot, and notes that vehicles WITH autopilot have 40% fewer accidents than those without. This seems like very clean data, as it doesn't actually consider how often autopilot is turned on, and it's over a fairly large sample of drivers, locales and miles driven. This is just correlation, not causality, but the effect is too big to be by chance.

dwaltrip 9y ago
The comment you are replying to is suggesting that the miles driven with autopilot may have been inherently safer miles, e.g. autopilot was more likely to be turned during the safer parts of a drive. If true, this would be an unfortunate example of selection bias.
- URSpider94 9y ago
  You are missing the point. The report doesn't compare miles driven with and without autopilot. It compares ALL miles driven in cars with Autopilot function, whether it is turned on or not, vs. cars without Autopilot function. To boot, these are, for the most part, the same cars before and after, since Tesla shipped cars for almost a year with the Autopilot hardware built in but not activated, then activated it for all cars at virtually the same time via an OTA. Before that date, any miles clocked show up in the "no autopilot" column, the next day they show up in the "with Autopilot" column. To further flog an expired equine, the study showed a 40% reduction in airbag actuations per million miles driven across all miles driven in cars with vs. without Autopilot function, not comparing only miles where Autopilot was turned on. There are certainly possibilities for this data to be skewed or biased, but it's definitely not a comparison of safe Autopilot miles vs. unsafe manual miles.