3 ms·

If it were the case that autopilot was benefiting from dumping all of the difficult parts on the humans, you would expect to see much higher than average rates

If it were the case that autopilot was benefiting from dumping all of the difficult parts on the humans, you would expect to see much higher than average rates of accidents per mile on the human driven parts, because they're only getting the hard parts. This tends not to be true, so I'm doubting it's a real effect.

mrtksn 5y ago
Humans don’t disengage and let the machines handle the situation but machines do it all the time. Do the machines disengage at tricky situations or straight lines?
- spywaregorilla 5y ago
  The statistics are there to read however, and suggest your reasoning is not correct. Case A: Non Tesla Human drives 999 easy miles and 1 hard mile Case B: Autopilot drives 999 easy miles, and human drives 1 hard mile If the effect size of the hard mile is so large that its skewing the statistics, you would expect the Telsa human driver to have a horrendous per mile accident rate relative to the non tesla human driver. What's most likely is that the accidents following a human handoff get correctly allocated to the Autopilot and the effect size being described is not actually that significant.
  - mrtksn 5y ago
    stats for easy miles and hard miles? link please? Humans don’t crash at every tricky situation but Tesla claims that humans are horrible drivers and their cars gives the control to the humans when when something happens.
    - spywaregorilla 5y ago
      Literally any distribution of hard miles and easy miles will produce the same outcome in this thought experiment, to varying effects, if your premise of Tesla hiding the AI's incompetence with passing off to humans when driving is challenging is accurate. If your premise is to be believed as a significant effect, you must also accept that this outcome should be visible in the data
  - buran77 5y ago
    Student drivers (with an instructor in the car, and secondary controls) have very few accidents, if any, not because of their safety record but because every mistake is saved by the instructor. A car can be considered as driving "safer than a human" when it can match the average human in any conditions the average human drives. Everything else is just squinting at the data and choosing an interpretation that fits your personal opinion. A Tesla can relatively safely cover traffic like divided or controlled access highways. That's a very narrow slice of all possible driving situations and not one responsible for most accidents.
    - spywaregorilla 5y ago
      So as a thought experiment, if Autopilot + Human intervention reduced the rate of accidents by 50% vs. just humans after normalization; can we consider autopilot to be adding value?
      - buran77 5y ago
        Of course the Autopilot adds value, all such driver assists are there to add value and help the driver on any car. I just don't think Autopilot can drive, let alone drive safer than a human. There's a difference between "helping a human drive safer" and "driving safer than a human". This is a confusion many people make when reading these stats, to Tesla's benefit. Also the oldest Tesla with AP is less than 10 years old (with an average of 3-4 years given the sales trends). The average age of cars on the street in the US is over 12 years old. An accident statistic that looks at reasonably new, premium cars against everything else won't paint the correct picture.
    - Dylan16807 5y ago
      If that means the supervision actually works, then it doesn't sound like there's a problem?
    - runarberg 5y ago
      > Student drivers (with an instructor in the car, and secondary controls) have very few accidents [...] because every mistake is saved by the instructor Is that the only reason? I suspect that many (most) student drivers are also in general more alert, drive slower, and follow each rule/guideline to an extreme. If they don’t the instructor will probably end the lesson and remove a dangerous driver of the road.
  - mkipper 5y ago
    Is this data available anywhere? How do I know that non-autopilot Tesla mile per accident rates aren't horrible? Tesla publishes their own data for the safety of autopilot, which I presume is based on their own analysis of accident records. Is this same detailed information available to other groups (insurers, NIST, etc)? Or do they just calculate an aggregate "Tesla mile per accident" rate that is a blend of the great autopilot rate and horrible human rate? I'm not trying to be facetious here. I have no idea if this data is available to any groups other than Tesla themselves. And if so, do they publish those numbers?
    - spywaregorilla 5y ago
      Tesla releases the miles per crash rates quarterly, for autopilot and non autopilot cases. Autopilot crashes include anything within 5 seconds of disengagement. The human rate tends to be more than 2x worse than the autopilot rate. This is not normalized for factors like road context. The human rate for tesla driven miles tends to be ~4x better than the other brands' average. To precisely answer this question you would want to see both a comparable brand's humans' performance; and probably the split of humans who used autopilot and humans who don't ever use autopilot. We don't have that, but in my personal opinion there's enough evidence to suggest it's probably not a grand conspiracy. I'm of the opinion that autopilot being ballpark on par with other drivers is more than enough to reduce accidents substantially, at scale. https://www.tesla.com/VehicleSafetyReport https://www.tesla.com/VehicleSafetyReport
  - jonathankoren 5y ago
    It’s not just that. There’s a context switch when the autodrive disengages, so the human is actually less ready for the hard mile than if they were driving the whole time. Sure m, the human is supposed to be able to take control at anytime, but I don’t think that really happens. The whole purpose of autodrive is do you don’t have to pay attention and drive.
jcranberry 5y ago
Could you provide a source? That sounds fairly interesting.
- spywaregorilla 5y ago
  Just some reasoning really. Statistics and proper normalization are hard. Tesla tends to say that autopilot crashes occur 1/2 as often as non autopilot crashes. That's likely not normalized to road conditions. But if you assume that Tesla is secretly just putting all the hard miles on the humans, then that would imply humans are driving many more hard miles and should have higher accident rates. The autopilots meanwhile must be performing worse on the easy miles and racking up additional accidents that wouldn't have otherwise happened. If you combine those two, the overall rate of accidents should be higher than average, but it's actually lower by a fair margin. Again, normalization is hard. Ideally you would be able to compare human drivers of another comparable car brand to the human drivers of Tesla to confirm the Tesla drivers don't seem to be being judged on unreasonably difficult conditions.
- mrtksn 5y ago
  There was a sourse but I could not find it ATM. It’s fairly simple, people don’t disengage and their driving safety is judged over all the miles they drive + all the situations where Autopilot disengages. Tesla Autopilot is judged only by the miles driven without disengagement, which is quite limited actually. You can watch Youtube videos to see at what kind of situations Tesla autopilot gives up. There’s no situation where the Autopilot takes over from the human saying “That’s a tricky road, let me handle it”.
  - spywaregorilla 5y ago
    You seem to be missing the point though. If this were significant, then human tesla drivers should be shown as performing much worse than other car drivers, because you're claiming they have a disproportionately large riding time in "tricky roads". A non tesla driver should be doing way better because they get to pad their score with the easy roads the autopilot supposedly gets.
FireBeyond 5y ago
Your conclusion relies on the - quite faulty - assumption that "situations that inherently difficult for FSD to handle" are automatically "also more dangerous for human drivers". In snowy conditions, humans do just fine, generally, at following "lanes", be they the actual lane, or the safest route that everyone else is following. Humans are also capable of deducting lane direction, orientation, even when there are contradictory/old lane markings on the road, a situation FSD regularly causes danger in. Or that that negative effect is lost in the orders of magnitude of "all human drivers across all miles" versus FSD.
- tshaddox 5y ago
  You don't think that human driver accident rates in snowy conditions are much higher than in fair weather conditions?
  - FireBeyond 5y ago
    I was referring to the lane "following" aspect. FSD disengages, but most humans can follow lanes in snow. Of course there are separate concerns.

mrtksn 5y ago

Humans don’t disengage and let the machines handle the situation but machines do it all the time. Do the machines disengage at tricky situations or straight lines?

spywaregorilla 5y ago
The statistics are there to read however, and suggest your reasoning is not correct. Case A: Non Tesla Human drives 999 easy miles and 1 hard mile Case B: Autopilot drives 999 easy miles, and human drives 1 hard mile If the effect size of the hard mile is so large that its skewing the statistics, you would expect the Telsa human driver to have a horrendous per mile accident rate relative to the non tesla human driver. What's most likely is that the accidents following a human handoff get correctly allocated to the Autopilot and the effect size being described is not actually that significant.
- mrtksn 5y ago
  stats for easy miles and hard miles? link please? Humans don’t crash at every tricky situation but Tesla claims that humans are horrible drivers and their cars gives the control to the humans when when something happens.
  - spywaregorilla 5y ago
    Literally any distribution of hard miles and easy miles will produce the same outcome in this thought experiment, to varying effects, if your premise of Tesla hiding the AI's incompetence with passing off to humans when driving is challenging is accurate. If your premise is to be believed as a significant effect, you must also accept that this outcome should be visible in the data
- buran77 5y ago
  Student drivers (with an instructor in the car, and secondary controls) have very few accidents, if any, not because of their safety record but because every mistake is saved by the instructor. A car can be considered as driving "safer than a human" when it can match the average human in any conditions the average human drives. Everything else is just squinting at the data and choosing an interpretation that fits your personal opinion. A Tesla can relatively safely cover traffic like divided or controlled access highways. That's a very narrow slice of all possible driving situations and not one responsible for most accidents.
  - spywaregorilla 5y ago
    So as a thought experiment, if Autopilot + Human intervention reduced the rate of accidents by 50% vs. just humans after normalization; can we consider autopilot to be adding value?
    - buran77 5y ago
      Of course the Autopilot adds value, all such driver assists are there to add value and help the driver on any car. I just don't think Autopilot can drive, let alone drive safer than a human. There's a difference between "helping a human drive safer" and "driving safer than a human". This is a confusion many people make when reading these stats, to Tesla's benefit. Also the oldest Tesla with AP is less than 10 years old (with an average of 3-4 years given the sales trends). The average age of cars on the street in the US is over 12 years old. An accident statistic that looks at reasonably new, premium cars against everything else won't paint the correct picture.
  - Dylan16807 5y ago
    If that means the supervision actually works, then it doesn't sound like there's a problem?
  - runarberg 5y ago
    > Student drivers (with an instructor in the car, and secondary controls) have very few accidents [...] because every mistake is saved by the instructor Is that the only reason? I suspect that many (most) student drivers are also in general more alert, drive slower, and follow each rule/guideline to an extreme. If they don’t the instructor will probably end the lesson and remove a dangerous driver of the road.
- mkipper 5y ago
  Is this data available anywhere? How do I know that non-autopilot Tesla mile per accident rates aren't horrible? Tesla publishes their own data for the safety of autopilot, which I presume is based on their own analysis of accident records. Is this same detailed information available to other groups (insurers, NIST, etc)? Or do they just calculate an aggregate "Tesla mile per accident" rate that is a blend of the great autopilot rate and horrible human rate? I'm not trying to be facetious here. I have no idea if this data is available to any groups other than Tesla themselves. And if so, do they publish those numbers?
  - spywaregorilla 5y ago
    Tesla releases the miles per crash rates quarterly, for autopilot and non autopilot cases. Autopilot crashes include anything within 5 seconds of disengagement. The human rate tends to be more than 2x worse than the autopilot rate. This is not normalized for factors like road context. The human rate for tesla driven miles tends to be ~4x better than the other brands' average. To precisely answer this question you would want to see both a comparable brand's humans' performance; and probably the split of humans who used autopilot and humans who don't ever use autopilot. We don't have that, but in my personal opinion there's enough evidence to suggest it's probably not a grand conspiracy. I'm of the opinion that autopilot being ballpark on par with other drivers is more than enough to reduce accidents substantially, at scale. https://www.tesla.com/VehicleSafetyReport https://www.tesla.com/VehicleSafetyReport
- jonathankoren 5y ago
  It’s not just that. There’s a context switch when the autodrive disengages, so the human is actually less ready for the hard mile than if they were driving the whole time. Sure m, the human is supposed to be able to take control at anytime, but I don’t think that really happens. The whole purpose of autodrive is do you don’t have to pay attention and drive.

jcranberry 5y ago

Could you provide a source? That sounds fairly interesting.

spywaregorilla 5y ago
Just some reasoning really. Statistics and proper normalization are hard. Tesla tends to say that autopilot crashes occur 1/2 as often as non autopilot crashes. That's likely not normalized to road conditions. But if you assume that Tesla is secretly just putting all the hard miles on the humans, then that would imply humans are driving many more hard miles and should have higher accident rates. The autopilots meanwhile must be performing worse on the easy miles and racking up additional accidents that wouldn't have otherwise happened. If you combine those two, the overall rate of accidents should be higher than average, but it's actually lower by a fair margin. Again, normalization is hard. Ideally you would be able to compare human drivers of another comparable car brand to the human drivers of Tesla to confirm the Tesla drivers don't seem to be being judged on unreasonably difficult conditions.
mrtksn 5y ago
There was a sourse but I could not find it ATM. It’s fairly simple, people don’t disengage and their driving safety is judged over all the miles they drive + all the situations where Autopilot disengages. Tesla Autopilot is judged only by the miles driven without disengagement, which is quite limited actually. You can watch Youtube videos to see at what kind of situations Tesla autopilot gives up. There’s no situation where the Autopilot takes over from the human saying “That’s a tricky road, let me handle it”.
- spywaregorilla 5y ago
  You seem to be missing the point though. If this were significant, then human tesla drivers should be shown as performing much worse than other car drivers, because you're claiming they have a disproportionately large riding time in "tricky roads". A non tesla driver should be doing way better because they get to pad their score with the easy roads the autopilot supposedly gets.

FireBeyond 5y ago

Your conclusion relies on the - quite faulty - assumption that "situations that inherently difficult for FSD to handle" are automatically "also more dangerous for human drivers". In snowy conditions, humans do just fine, generally, at following "lanes", be they the actual lane, or the safest route that everyone else is following. Humans are also capable of deducting lane direction, orientation, even when there are contradictory/old lane markings on the road, a situation FSD regularly causes danger in. Or that that negative effect is lost in the orders of magnitude of "all human drivers across all miles" versus FSD.

tshaddox 5y ago
You don't think that human driver accident rates in snowy conditions are much higher than in fair weather conditions?
- FireBeyond 5y ago
  I was referring to the lane "following" aspect. FSD disengages, but most humans can follow lanes in snow. Of course there are separate concerns.