13 ms·

> The safety agency informed Tesla weeks ago that it was looking into the spate of crashes in which Tesla vehicles operating on Autopilot failed to detect stopp

> The safety agency informed Tesla weeks ago that it was looking into the spate of crashes in which Tesla vehicles operating on Autopilot failed to detect stopped emergency vehicles with flashing lights. The regulator originally said it was looking into 11 such crashes. A 12th occurred on Saturday, when a Model 3 hit a police cruiser that had stopped behind a car that had broken down on an interstate in Orlando, Fla. 12 crashes into vehicles with flashing lights on them doesn't sound like great success from Autopilot, gonna be honest.

brianwawok 5y ago
The driver is responsible for stopping the car. You literally agree to it when you enable AP. People are too comfortable and not paying attention.
- mhermher 5y ago
  The fact that it occurs so often should suggest to you that autopilot itself makes people too comfortable and not pay attention.
  - jerf 5y ago
    I, and many others, take the position that there really isn't any practical gradation between more-or-less conventional driving, maybe with light lane switching assistance or something, and full on "Level 5" autonomous driving, because the idea that human brains can watch something for hours at a time only to take instant, sub-second actions every few hours is an insane model of human cognition. I don't disagree with the idea of having a gradient to measure our progress up to full autonomy, but I haven't seen much to say that we should deploy the intermediate stages. Until autonomy can be reasonably guaranteed at at least human levels of safety under the assumption of no human control, it really shouldn't be deployed. "But the human had 673 milliseconds to take control and avert the accident! It's their fault!" is simply not a reasonable defense.
    - joelwilliamson 5y ago
      Shouldn't the objection be to anything less than Level 4, rather than Level 5? Level 4 requires that the vehicle can safely stop itself and wait for a human to take over when it is faced with a situation it can't handle. (There's an argument that if you take away the routine parts of driving, humans won't be able to deal with anomalous situations, but that's a different argument than the attentiveness one.)
      - jerf 5y ago
        If that is possible, yes. I'm not convinced it is possible to create an autonomous vehicle that only panics in situations where there is a safe pull-over, or, if such a vehicle exists, that it'll be considered "autonomous" when it can't go past 25 or 35 mph or something. There's plenty of driving situations that may be confusing where pulling over and stopping isn't possible. (And, to be clear, I'm not asking for perfect safety, I'm asking for "human or better", and also understand that there may be some interesting fuzziness near the border where the two have different strengths and weakenesses but come out statistically the same. However, I don't feel like we're there yet.)
  - galangalalgol 5y ago
    Mixed control between automated and human for tasks that require persistent attention is bad. We have known this for decades. The example often given in engineering textbooks is 3 mile island. I suspect that the engineers involved in self driving systems know this too, but the liability is too great to make the switch all at once. It seems better to take baby steps into self driving, but history teaches us it should be all or nothing.
    - fallingknife 5y ago
      You could say the same about airplane autopilots, but they have an amazing safety record.
      - kaibee 5y ago
        Airplanes also have 2 pilots and nothing to crash into.
      - Piskvorrr 5y ago
        Far fewer objects to hit in the air: disengagement means "keep going this way," rarely "and crash into an eventual obstacle". Plus, their early safety record is written in blood like everything else is.
        fallingknife 5y ago
        > Plus, their early safety record is written in blood like everything else is. Yeah, and that's how it's going to be with car autopilots. Which is why it pisses me off that people want to stop progress on it.
        Piskvorrr 5y ago
        Interesting typoes in "be unwilling test subjects". Feel free to volunteer, though. Elaine Herzberg sure didn't.
      - meepmorp 5y ago
- deleted 5y ago
  [deleted]
- meepmorp 5y ago
  > The driver is responsible for stopping the car. You literally agree to it when you enable AP. People are too comfortable and not paying attention. Autopilot encourages people to not pay as much attention while driving, until a critical situation comes up at which point they're expected to be fully in control. That's a design flaw that isn't fixed with a liability shifting "Agree" button.
  - tyfon 5y ago
    It will nag you every 17 seconds (if not attentive/holding the wheel with a torque) and abort auto pilot completely if you do not comply. If you do this three times in a drive you have to power off and on the car again to be able to use autopilot again. I've done maybe 40.000 km on autopilot and it's very aggressive compared to my other car (ioniq electric) when it comes to attention. The fact that some people purposely defeat this system with weights on the wheel is another matter. But saying that tesla encourages you to not paying attention is not correct.
    - jsight 5y ago
      To be fair, the person you are responding to may not like those other systems either. All the articles with industry analysts insisting that other driver monitoring systems are nearly perfect have sometimes blinded me to that fact. A lot of people don't like any of these systems for a lot of the same reasons that people didn't like basic cruise control in the early days.
      - ghaff 5y ago
        I haven't used basic cruise control in years. It's actually convenient not to have to pay attention to the speed but I find, on roads with even moderate traffic, it comes at the cost of prioritizing maintaining a constant speed rather than flowing with traffic and generally staying in a lane.
- avereveard 5y ago
  You have no telemetry and no idea what actually happened, car might have veered without enough warning. Which is the whole point of having an investigation, really.
capableweb 5y ago
> 12 crashes into vehicles with flashing lights on them doesn't sound like great success from Autopilot, gonna be honest. I'm sorry, but how can you form an opinion so quickly? 12 crashes out of what? Any number? 12 crashes out of 10000000 scenarios sound like a pretty good record, while 12 crashes out of 15 scenarios indeed doesn't sound like a great success. But what information do you have that the rest of us doesn't?
- swiley 5y ago
  yes. 12 crashes out of the maximum possible number of encounters.
- AshamedCaptain 5y ago
  I still remember the days when the SciFi of the day sold robotic drivers as "they don't get tired, they don't make mistakes, they are relentless in their pursuit". Instead of that, we got robots that are apparently unable to avoid crashing into large vehicles with flashing lights. Sometimes. As usual, the future never ceases to disappoint me.
  - leesec 5y ago
    Well if you're basing your expectations of reality on SciFi books than I expect you will be dissapointed quite often.
    - Gravityloss 5y ago
      There's always dystopian scifi as well.
      - sslayer 5y ago
        Yes! - This has not failed to disappoint, rather it has exceeded all expectations!
  - DennisP 5y ago
    Maybe we just haven't reached that future yet.
brk 5y ago
We just did a 1,500 mile round-trip journey to visit family. Because of the Tesla stuff, I was casually aware of roadside emergency vehicles moreso than usual. We passed 28 police/fire/emergency vehicles that were next to our travel lane. I saw a number of Tesla vehicles on the road as well, each passing these same emergency vehicles, but not always in the same lane. By this point, there must be 1M+ incidents of Tesla vehicles passing roadside emergency vehicles in an adjacent lane. 12 crashes is 12 crashes too many, statistically speaking, but it's also a tiny tiny percentage of the total number of cases. IMO, it is good that there is federal investigation into this. My main concern is that is seems our government agencies at large are not very good when it comes to properly understanding anything technology related, or making relevant decisions/rulings/etc. on these kinds of things.
- fallingknife 5y ago
  > IMO, it is good that there is federal investigation into this > seems our government agencies at large are not very good when it comes to properly understanding anything technology related, or making relevant decisions/rulings/etc. on these kinds of things. Not sure how you think it's good then.
- tw04 5y ago
  >I saw a number of Tesla vehicles on the road as well, each passing these same emergency vehicles, but not always in the same lane. How do you know all of them were on autopilot? >12 crashes is 12 crashes too many, statistically speaking, but it's also a tiny tiny percentage of the total number of cases. It's also 12 more than any normal attentive human has gotten into, and an average attentive human driver is the absolute floor for self-driving on public roads unless Tesla is going to go to the lengths GM has to ensure the person behind the wheel is remaining attentive. >IMO, it is good that there is federal investigation into this. My main concern is that is seems our government agencies at large are not very good when it comes to properly understanding anything technology related, or making relevant decisions/rulings/etc. on these kinds of things. I think they're good at understanding if Tesla loses complete public confidence in self driving that is a net-loss for humanity and technology, as well as a prime source of political shitstorm when people ask why something wasn't done sooner.
nashashmi 5y ago
The fact that Tesla and Elon have been fighting these autopilot errors in completely the wrong way is the primary reason the feds should be involved. I have been using autopilot in Nissan. If the car were to crash into an ambulance light going off I would not be able to predict it and stop it from happening quickly enough. I just have a whole lot of trust in some basic things the car does. And this is one of those cases where they should have been able to get this right after the first crash but they didn’t and now the feds have to get involved. A lesson on why we can’t have nice things.
- CultMember001 5y ago
  We should not allow the Fed to stifle innovation and technological progress. If we are not willing to take risks, then we will never be able to reach out true potential of becoming a truly advanced space faring civilization and a multi-planetary species. The NHTSA does not have the technological capability to understand advanced AI used in Autopilot and Full Self Driving and they should stop their interference immediately. I have Tesla and I bought the Full Self Driving package. I believe it is far more valuable than the current price. I am willing to pay 30 times more for Full Self Driving.
noslenwerdna 5y ago
Thanks for your honesty. Hopefully the NHTSA is also going to investigate the many other cars with traffic aware cruise control, which have the same problem with stationary vehicles. I was driving my Rav4 with TACC enabled, and it would have crashed into a column of stopped cars if I hadn't noticed in time and intervened.
- ahahahahah 5y ago
  Tesla has something like 1000x greater rate of hitting stationary emergency vehicles. The fact that all the other cars have the same issue with tacc actually points even more strongly to something being wrong with Tesla.
  - kwhitefoot 5y ago
    > 1000x greater rate What's your denominator? Is this per year or per kilometre driven with TACC active?
  - noslenwerdna 5y ago
    citation, please.
tablespoon 5y ago
>> The safety agency informed Tesla weeks ago that it was looking into the spate of crashes in which Tesla vehicles operating on Autopilot failed to detect stopped emergency vehicles with flashing lights. The regulator originally said it was looking into 11 such crashes. A 12th occurred on Saturday, when a Model 3 hit a police cruiser that had stopped behind a car that had broken down on an interstate in Orlando, Fla. > 12 crashes into vehicles with flashing lights on them doesn't sound like great success from Autopilot, gonna be honest. It's really surprising what people will put up with. IIRC, recently "autopilot" had some regressions in the situations it could handle, and Elon Musk tweeted something to the effect of "Yeah, this version sucks since we're retraining all our NNs." Like how did something that half-baked ever make it to production in a safety critical system?

brianwawok 5y ago

The driver is responsible for stopping the car. You literally agree to it when you enable AP. People are too comfortable and not paying attention.

mhermher 5y ago
The fact that it occurs so often should suggest to you that autopilot itself makes people too comfortable and not pay attention.
- jerf 5y ago
  I, and many others, take the position that there really isn't any practical gradation between more-or-less conventional driving, maybe with light lane switching assistance or something, and full on "Level 5" autonomous driving, because the idea that human brains can watch something for hours at a time only to take instant, sub-second actions every few hours is an insane model of human cognition. I don't disagree with the idea of having a gradient to measure our progress up to full autonomy, but I haven't seen much to say that we should deploy the intermediate stages. Until autonomy can be reasonably guaranteed at at least human levels of safety under the assumption of no human control, it really shouldn't be deployed. "But the human had 673 milliseconds to take control and avert the accident! It's their fault!" is simply not a reasonable defense.
  - joelwilliamson 5y ago
    Shouldn't the objection be to anything less than Level 4, rather than Level 5? Level 4 requires that the vehicle can safely stop itself and wait for a human to take over when it is faced with a situation it can't handle. (There's an argument that if you take away the routine parts of driving, humans won't be able to deal with anomalous situations, but that's a different argument than the attentiveness one.)
    - jerf 5y ago
      If that is possible, yes. I'm not convinced it is possible to create an autonomous vehicle that only panics in situations where there is a safe pull-over, or, if such a vehicle exists, that it'll be considered "autonomous" when it can't go past 25 or 35 mph or something. There's plenty of driving situations that may be confusing where pulling over and stopping isn't possible. (And, to be clear, I'm not asking for perfect safety, I'm asking for "human or better", and also understand that there may be some interesting fuzziness near the border where the two have different strengths and weakenesses but come out statistically the same. However, I don't feel like we're there yet.)
- galangalalgol 5y ago
  Mixed control between automated and human for tasks that require persistent attention is bad. We have known this for decades. The example often given in engineering textbooks is 3 mile island. I suspect that the engineers involved in self driving systems know this too, but the liability is too great to make the switch all at once. It seems better to take baby steps into self driving, but history teaches us it should be all or nothing.
  - fallingknife 5y ago
    You could say the same about airplane autopilots, but they have an amazing safety record.
    - kaibee 5y ago
      Airplanes also have 2 pilots and nothing to crash into.
    - Piskvorrr 5y ago
      Far fewer objects to hit in the air: disengagement means "keep going this way," rarely "and crash into an eventual obstacle". Plus, their early safety record is written in blood like everything else is.
      - fallingknife 5y ago
        > Plus, their early safety record is written in blood like everything else is. Yeah, and that's how it's going to be with car autopilots. Which is why it pisses me off that people want to stop progress on it.
        Piskvorrr 5y ago
        Interesting typoes in "be unwilling test subjects". Feel free to volunteer, though. Elaine Herzberg sure didn't.
    - meepmorp 5y ago
deleted 5y ago
[deleted]
meepmorp 5y ago
> The driver is responsible for stopping the car. You literally agree to it when you enable AP. People are too comfortable and not paying attention. Autopilot encourages people to not pay as much attention while driving, until a critical situation comes up at which point they're expected to be fully in control. That's a design flaw that isn't fixed with a liability shifting "Agree" button.
- tyfon 5y ago
  It will nag you every 17 seconds (if not attentive/holding the wheel with a torque) and abort auto pilot completely if you do not comply. If you do this three times in a drive you have to power off and on the car again to be able to use autopilot again. I've done maybe 40.000 km on autopilot and it's very aggressive compared to my other car (ioniq electric) when it comes to attention. The fact that some people purposely defeat this system with weights on the wheel is another matter. But saying that tesla encourages you to not paying attention is not correct.
  - jsight 5y ago
    To be fair, the person you are responding to may not like those other systems either. All the articles with industry analysts insisting that other driver monitoring systems are nearly perfect have sometimes blinded me to that fact. A lot of people don't like any of these systems for a lot of the same reasons that people didn't like basic cruise control in the early days.
    - ghaff 5y ago
      I haven't used basic cruise control in years. It's actually convenient not to have to pay attention to the speed but I find, on roads with even moderate traffic, it comes at the cost of prioritizing maintaining a constant speed rather than flowing with traffic and generally staying in a lane.
avereveard 5y ago
You have no telemetry and no idea what actually happened, car might have veered without enough warning. Which is the whole point of having an investigation, really.

capableweb 5y ago

> 12 crashes into vehicles with flashing lights on them doesn't sound like great success from Autopilot, gonna be honest. I'm sorry, but how can you form an opinion so quickly? 12 crashes out of what? Any number? 12 crashes out of 10000000 scenarios sound like a pretty good record, while 12 crashes out of 15 scenarios indeed doesn't sound like a great success. But what information do you have that the rest of us doesn't?

swiley 5y ago
yes. 12 crashes out of the maximum possible number of encounters.
AshamedCaptain 5y ago
I still remember the days when the SciFi of the day sold robotic drivers as "they don't get tired, they don't make mistakes, they are relentless in their pursuit". Instead of that, we got robots that are apparently unable to avoid crashing into large vehicles with flashing lights. Sometimes. As usual, the future never ceases to disappoint me.
- leesec 5y ago
  Well if you're basing your expectations of reality on SciFi books than I expect you will be dissapointed quite often.
  - Gravityloss 5y ago
    There's always dystopian scifi as well.
    - sslayer 5y ago
      Yes! - This has not failed to disappoint, rather it has exceeded all expectations!
- DennisP 5y ago
  Maybe we just haven't reached that future yet.

brk 5y ago

We just did a 1,500 mile round-trip journey to visit family. Because of the Tesla stuff, I was casually aware of roadside emergency vehicles moreso than usual. We passed 28 police/fire/emergency vehicles that were next to our travel lane. I saw a number of Tesla vehicles on the road as well, each passing these same emergency vehicles, but not always in the same lane. By this point, there must be 1M+ incidents of Tesla vehicles passing roadside emergency vehicles in an adjacent lane. 12 crashes is 12 crashes too many, statistically speaking, but it's also a tiny tiny percentage of the total number of cases. IMO, it is good that there is federal investigation into this. My main concern is that is seems our government agencies at large are not very good when it comes to properly understanding anything technology related, or making relevant decisions/rulings/etc. on these kinds of things.

fallingknife 5y ago
> IMO, it is good that there is federal investigation into this > seems our government agencies at large are not very good when it comes to properly understanding anything technology related, or making relevant decisions/rulings/etc. on these kinds of things. Not sure how you think it's good then.
tw04 5y ago
>I saw a number of Tesla vehicles on the road as well, each passing these same emergency vehicles, but not always in the same lane. How do you know all of them were on autopilot? >12 crashes is 12 crashes too many, statistically speaking, but it's also a tiny tiny percentage of the total number of cases. It's also 12 more than any normal attentive human has gotten into, and an average attentive human driver is the absolute floor for self-driving on public roads unless Tesla is going to go to the lengths GM has to ensure the person behind the wheel is remaining attentive. >IMO, it is good that there is federal investigation into this. My main concern is that is seems our government agencies at large are not very good when it comes to properly understanding anything technology related, or making relevant decisions/rulings/etc. on these kinds of things. I think they're good at understanding if Tesla loses complete public confidence in self driving that is a net-loss for humanity and technology, as well as a prime source of political shitstorm when people ask why something wasn't done sooner.

nashashmi 5y ago

The fact that Tesla and Elon have been fighting these autopilot errors in completely the wrong way is the primary reason the feds should be involved. I have been using autopilot in Nissan. If the car were to crash into an ambulance light going off I would not be able to predict it and stop it from happening quickly enough. I just have a whole lot of trust in some basic things the car does. And this is one of those cases where they should have been able to get this right after the first crash but they didn’t and now the feds have to get involved. A lesson on why we can’t have nice things.

CultMember001 5y ago
We should not allow the Fed to stifle innovation and technological progress. If we are not willing to take risks, then we will never be able to reach out true potential of becoming a truly advanced space faring civilization and a multi-planetary species. The NHTSA does not have the technological capability to understand advanced AI used in Autopilot and Full Self Driving and they should stop their interference immediately. I have Tesla and I bought the Full Self Driving package. I believe it is far more valuable than the current price. I am willing to pay 30 times more for Full Self Driving.

noslenwerdna 5y ago

Thanks for your honesty. Hopefully the NHTSA is also going to investigate the many other cars with traffic aware cruise control, which have the same problem with stationary vehicles. I was driving my Rav4 with TACC enabled, and it would have crashed into a column of stopped cars if I hadn't noticed in time and intervened.

ahahahahah 5y ago
Tesla has something like 1000x greater rate of hitting stationary emergency vehicles. The fact that all the other cars have the same issue with tacc actually points even more strongly to something being wrong with Tesla.
- kwhitefoot 5y ago
  > 1000x greater rate What's your denominator? Is this per year or per kilometre driven with TACC active?
- noslenwerdna 5y ago
  citation, please.

tablespoon 5y ago

>> The safety agency informed Tesla weeks ago that it was looking into the spate of crashes in which Tesla vehicles operating on Autopilot failed to detect stopped emergency vehicles with flashing lights. The regulator originally said it was looking into 11 such crashes. A 12th occurred on Saturday, when a Model 3 hit a police cruiser that had stopped behind a car that had broken down on an interstate in Orlando, Fla. > 12 crashes into vehicles with flashing lights on them doesn't sound like great success from Autopilot, gonna be honest. It's really surprising what people will put up with. IIRC, recently "autopilot" had some regressions in the situations it could handle, and Elon Musk tweeted something to the effect of "Yeah, this version sucks since we're retraining all our NNs." Like how did something that half-baked ever make it to production in a safety critical system?