10 ms·

Current Tesla autopilot is sometimes blind to stopped cars

49 points by matt2000 8y ago

xedeon 8y ago
You mean Traffic-Aware Cruise Control (TACC)?
thecopy 8y ago
Its just a matter of time before the injured and dead count starts increasing. Tesla needs to take their responsibility and disable AutoPilot before more people die because of its severe faults and flaws.
- nickik 8y ago
  Currently the amount of injured and dead is falling for both Tesla cars and Tesla cars while on Autopilot.
  - michaelt 8y ago
    I didn't realise Tesla Autopilot could raise the dead :)
  - code_duck 8y ago
    You mean the rate? The total is very unlikely to ever decrease.
eastendguy 8y ago
Can any other Traffic-Aware Cruise Control (TACC) stop in such a case? (Real question, I don't know). My current understanding is that Tesla's TACC is the best on the market. Correct?
- toomuchtodo 8y ago
  This is a deficiency in several automatic emergency braking systems [1]. Tesla extracts as much signal from front facing radar as they can, but edge cases abound. Marketplace or Planet Money had a piece a few weeks ago that mentioned the hardest problem was consumer education about these safety systems. People are inherently not interested in learning about the life safety systems in their cars. They expect it to “just work”. Whose failing is it if warnings clearly state that isn’t the case? “The car companies don't hide the fact that today's AEB systems have blind spots. It's all there in the owner's manuals, typically covered by both an all-encompassing legal disclaimer and explicit examples of why the systems might fail to intervene. For instance, the Camry's AEB system may not work when you're driving on a hill. It might not spot vehicles with high ground clearance or those with low rear ends. It may not work if a wiper blade blocks the camera. Toyota says the system could also fail if the vehicle is wobbling, whatever that means. It may not function when the sun shines directly on the vehicle ahead or into the camera mounted near the rearview mirror.” [1] https://www.caranddriver.com/features/a24511826/safety-features-automatic-braking-system-tested-explained/ https://www.caranddriver.com/features/a24511826/safety-featu...
  - jeremyjh 8y ago
    Without doubt it is Tesla’s fault, due to how they have named and marketed the feature. Even if they fixed the name, they have created a system that encourages people to disengage when it is not safe to do so. You can’t make a system inherently unsafe and absolve responsibility by adding a warning.
    - toomuchtodo 8y ago
      Tesla isn’t responsible for ignorant consumers. The lowest common denominator should not guide product marketing and education.
  - VikingCoder 8y ago
    > Whose failing is it if warnings clearly state that isn’t the case? Tesla should be responsible and stop marketing it. Otherwise you're begging legislators to write ham-handed laws, begging courts to set awful precedents. Either way, they are absolutely poisoning the well.
- baddox 8y ago
  I don’t think so. My understanding is that this is completely intended (and documented) behavior in Teslas and all other makes with this feature. Correct me if I’m wrong, but no adaptive cruise control systems perform full emergency breaking or emergency swerving when traveling at high speeds. Here’s an an article from last year: https://www.wired.com/story/tesla-autopilot-why-crash-radar/ https://www.wired.com/story/tesla-autopilot-why-crash-radar/
  - nradov 8y ago
    The latest generation of Volvo's driver assistance system will do emergency braking and swerving at high speeds. https://www.caranddriver.com/news/a15342750/2018-volvo-xc60-will-help-drivers-steer-clear-of-trouble/ https://www.caranddriver.com/news/a15342750/2018-volvo-xc60-...
- superice 8y ago
  I drive a 2018 VW Golf Variant (Station Wagon) and the Adaptive Cruise Control from VW also has this problem. I regularly use it for city driving from traffic light to traffic light which means that I basically only have to steer a little, but if I'm driving towards a stationary car waiting in front of a red light, it will not detect the car until it starts to move.
  - kijin 8y ago
    So it's like the T. Rex from the original Jurassic Park movie? "He can't see you if you don't move."
- mikejb 8y ago
  > Can any other Traffic-Aware Cruise Control (TACC) stop in such a case? (Real question, I don't know). I haven't found conclusive evidence, but the lack of news around other TACCs crashing into stationary objects is an indicator it might be a Autopilot specific issue. > My current understanding is that Tesla's TACC is the best on the market. Correct? Define best. It definitely has a lot of features few or no other systems have; on the other hand, it's a test program. It's officially Beta-Software, given to drivers to test and identify issues. So from a feature-set it's probably the best you can buy - but it turns you into a Test-Driver rather than a user of a product.
  - kokx 8y ago
    > I haven't found conclusive evidence, but the lack of news around other TACCs crashing into stationary objects is an indicator it might be a Autopilot specific issue. IMO, I think this may actually not be true. Due to Tesla's popularity, and the amount of investors that have stated that they were shorting TSLA, I do think that media have a tendency to report more news about such accidents just because a Tesla was involved. If a Tesla crashes somewhere, the first question probably is "Was Autopilot enabled?", while with most other car crashes, it's just a car crash. So I don't think that lack of news about other cars with TACC means that they don't suffer from similar issues. (TBH I wouldn't be able to name any other cars with such an advanced TACC either).
    - shereadsthenews 8y ago
      Tesla’s popularity? It is the twentieth-best-selling brand.
- VikingCoder 8y ago
  Sounds like TACC is deadly, then. Can Waymo handle this situation or not?
- rayiner 8y ago
  Tesla's system is not the best: https://www.caranddriver.com/features/a24511826/safety-features-automatic-braking-system-tested-explained https://www.caranddriver.com/features/a24511826/safety-featu....
jameskilton 8y ago
This has been true for a long time. Radar can't differentiate between a wall, a stopped car, or a dead skunk. Heck, it can get confused with a bridge's shadow on a bright day. Thus autopilot has to be configured to deal with these situations: stop or run into them. The vast majority of the situation you don't want the car to stop, as that will lead to a more dangerous situation and increase the chance of a collision. It's one of the many situations that show that "autopilot", particularly Level 4 and Level 5, have a very, very long way to go. Also, as the reddit thread discusses the psychology of autopilot, this is why I don't want self-driving cars yet. Humans are horrendous at paying attention when they aren't directly in control. Airlines have been dealing with this problem for decades. Tesla has even started calling their services "driver assist" instead of autopilot to help combat this problem. And yes, many people use autopilot without a problem every day, but you have to remember that it only takes one single miss to kill you (see: driver killed by ramming into a lane divider on autopilot).
- tntn 8y ago
  > Radar can't differentiate between a wall, a stopped car, or a dead skunk "Radar" absolutely can. What can't differentiate between these things is the radar in Teslas that was intended for use in adaptive cruise control. There are any number of not insanely expensive radars that could do this job perfectly fine. Tesla just decided to go "full self driving" with this rather bad radar because "humans can do it with vision only!!!1!!!1!"
  - Junk_Collector 8y ago
    I don't the specifics of Tesla's radar implementation but what I have seen of the accidents usually seem like a textbook examples of incorrectly implemented clutter filtering. It's not an easy problem to solve in practice (re: cheaply), but known solutions have existed for decades.
    - tntn 8y ago
      The radar module in Teslas uses the same filtering strategy as most adaptive cruise control radar modules, which is "if relative velocity to object is nearly the opposite of the car's velocity, throw it out." It works pretty much as designed.
stevoo 8y ago
This not news. This has existed from the first iteration of Tesla Autopilots. That is why there where crashes with stationery vehicles for Tesla. This is shortcoming for using only cameras or any vehicle that has adaptive cruise control. The problem is that the front camera cannot distinguish from the road and a stationery vehicle. For that, they are the same and they ignore stationary obstacles. If they where taken into consideration, then they would not work at all. This is where Lidar comes in but at its current price, or at price that Tesla was designing its system it was too expensive. I am not entirely sure that pure cameras will ever be enough
- derekp7 8y ago
  This is something I don't understand. If you have more than 1 camera, you can run an edge detection algorithm on the images from each camera and use simple trig to build up a 3d model of what is in front of the car. This can easily distinguish between a shadow or road kill, vs a large object in the way. Is there a technical reason they don't do this (such as too much processing power required)?
  - rcxdude 8y ago
    You are trivialising the problem. Getting reliable data from stereo images is still far from solved, especially in realtime. (Also, the best currently available techniques are likely too computationally expensive to run on the compute that Tesla has put in the car).
    - fabianluque 8y ago
      You can clearly see here (https://www.youtube.com/watch?v=ROHW3sm2j0g https://www.youtube.com/watch?v=ROHW3sm2j0g) that Autopilot is detecting moving, stationary, stopped and oncoming vehicles. There could be situations where this does not work accurately and that's when problems occur. If you watch closely you can even see it sometimes in this video.
      - kevin_thibedeau 8y ago
athenot 8y ago
Yet isn't this addressed by the "dumb" collision avoidance system found in many premium cars? ("dumb" as in limited in scope and not trying to drive the car as a whole.)
gthtjtkt 8y ago
What are we going to find out next, water is wet? Every radar cruise control system comes with a warning that it can't detect stopped traffic. This seems like a case of RTFM.
- tntn 8y ago
  The issue is that tesla packaged up a radar designed for adaptive cruise control as part of their "autopilot." Maybe there is some disclaimer somewhere that says autopilot will happily run into stopped cars, but I don't know that most drivers of Teslas are aware of this.
  - gthtjtkt 8y ago
    It's in the manual. In fact, the exact scenario in this video is on page 83 of the Model S manual.
js2 8y ago
Readers might find this piece (“Four Cars Repeatedly Crashed to Test the Latest Automatic Braking Safety Systems”) interesting: https://www.caranddriver.com/features/a24511826/safety-features-automatic-braking-system-tested-explained/ https://www.caranddriver.com/features/a24511826/safety-featu...
gopher2 8y ago
It's so bad that they call this feature "Autopilot". Taking a page out of the AT&T 5G Evolution playbook I guess.
xnyan 8y ago
A LIDAR array is a requirement for automated driving. It can operate in some conditions that radar cannot. lidar, radar and computer vision should all be used in concert for best effect and Tesla knows this, but chooses/chose not to in order to meet a price point. That choice may have been the right one, I don't know. Hacker news (and me too, to be honest) focuses on Tesla, but there are a lot of other players doing this and doing it better IMO.
benjohnson 8y ago
This is rather useful.. If I notice a Tesla behind me, I'll do my best not to come to an abrupt complete stop. Sounds like if I slow down gradually, it will give it enough data to slow down as well.
kwhitefoot 8y ago
This is not news to anyone who own a Tesla. Perhaps someone who drives a car with a different radar could tell us if any other real world systems (actually installed on mass market vehicles right now) does better.
rayiner 8y ago
Tesla's system is not only not "autopilot" (it will happily plow into stationary vehicles), it's not even the best adaptive cruise control on the market for dealing with stationary vehicles: https://www.caranddriver.com/features/a24511826/safety-features-automatic-braking-system-tested-explained https://www.caranddriver.com/features/a24511826/safety-featu.... In a recent Car & Driver test, Subaru beat Tesla in every single scenario involving fast approach to a stationary/slow-moving target (although all collided with the target at closing speeds >= 50 mph).
Tomminn 8y ago
Is this just because it's hard to distinguish using visual data when it's a 2-D image on the road (from the sun, intentional paint, a spill etc) and when its a 3-D object sitting upright on the road?

49 points by matt2000 8y ago

You mean Traffic-Aware Cruise Control (TACC)?

Its just a matter of time before the injured and dead count starts increasing. Tesla needs to take their responsibility and disable AutoPilot before more people die because of its severe faults and flaws.

nickik 8y ago
Currently the amount of injured and dead is falling for both Tesla cars and Tesla cars while on Autopilot.
- michaelt 8y ago
  I didn't realise Tesla Autopilot could raise the dead :)
- code_duck 8y ago
  You mean the rate? The total is very unlikely to ever decrease.

eastendguy 8y ago

Can any other Traffic-Aware Cruise Control (TACC) stop in such a case? (Real question, I don't know). My current understanding is that Tesla's TACC is the best on the market. Correct?

toomuchtodo 8y ago
This is a deficiency in several automatic emergency braking systems [1]. Tesla extracts as much signal from front facing radar as they can, but edge cases abound. Marketplace or Planet Money had a piece a few weeks ago that mentioned the hardest problem was consumer education about these safety systems. People are inherently not interested in learning about the life safety systems in their cars. They expect it to “just work”. Whose failing is it if warnings clearly state that isn’t the case? “The car companies don't hide the fact that today's AEB systems have blind spots. It's all there in the owner's manuals, typically covered by both an all-encompassing legal disclaimer and explicit examples of why the systems might fail to intervene. For instance, the Camry's AEB system may not work when you're driving on a hill. It might not spot vehicles with high ground clearance or those with low rear ends. It may not work if a wiper blade blocks the camera. Toyota says the system could also fail if the vehicle is wobbling, whatever that means. It may not function when the sun shines directly on the vehicle ahead or into the camera mounted near the rearview mirror.” [1] https://www.caranddriver.com/features/a24511826/safety-features-automatic-braking-system-tested-explained/ https://www.caranddriver.com/features/a24511826/safety-featu...
- jeremyjh 8y ago
  Without doubt it is Tesla’s fault, due to how they have named and marketed the feature. Even if they fixed the name, they have created a system that encourages people to disengage when it is not safe to do so. You can’t make a system inherently unsafe and absolve responsibility by adding a warning.
  - toomuchtodo 8y ago
    Tesla isn’t responsible for ignorant consumers. The lowest common denominator should not guide product marketing and education.
- VikingCoder 8y ago
  > Whose failing is it if warnings clearly state that isn’t the case? Tesla should be responsible and stop marketing it. Otherwise you're begging legislators to write ham-handed laws, begging courts to set awful precedents. Either way, they are absolutely poisoning the well.
baddox 8y ago
I don’t think so. My understanding is that this is completely intended (and documented) behavior in Teslas and all other makes with this feature. Correct me if I’m wrong, but no adaptive cruise control systems perform full emergency breaking or emergency swerving when traveling at high speeds. Here’s an an article from last year: https://www.wired.com/story/tesla-autopilot-why-crash-radar/ https://www.wired.com/story/tesla-autopilot-why-crash-radar/
- nradov 8y ago
  The latest generation of Volvo's driver assistance system will do emergency braking and swerving at high speeds. https://www.caranddriver.com/news/a15342750/2018-volvo-xc60-will-help-drivers-steer-clear-of-trouble/ https://www.caranddriver.com/news/a15342750/2018-volvo-xc60-...
superice 8y ago
I drive a 2018 VW Golf Variant (Station Wagon) and the Adaptive Cruise Control from VW also has this problem. I regularly use it for city driving from traffic light to traffic light which means that I basically only have to steer a little, but if I'm driving towards a stationary car waiting in front of a red light, it will not detect the car until it starts to move.
- kijin 8y ago
  So it's like the T. Rex from the original Jurassic Park movie? "He can't see you if you don't move."
mikejb 8y ago
> Can any other Traffic-Aware Cruise Control (TACC) stop in such a case? (Real question, I don't know). I haven't found conclusive evidence, but the lack of news around other TACCs crashing into stationary objects is an indicator it might be a Autopilot specific issue. > My current understanding is that Tesla's TACC is the best on the market. Correct? Define best. It definitely has a lot of features few or no other systems have; on the other hand, it's a test program. It's officially Beta-Software, given to drivers to test and identify issues. So from a feature-set it's probably the best you can buy - but it turns you into a Test-Driver rather than a user of a product.
- kokx 8y ago
  > I haven't found conclusive evidence, but the lack of news around other TACCs crashing into stationary objects is an indicator it might be a Autopilot specific issue. IMO, I think this may actually not be true. Due to Tesla's popularity, and the amount of investors that have stated that they were shorting TSLA, I do think that media have a tendency to report more news about such accidents just because a Tesla was involved. If a Tesla crashes somewhere, the first question probably is "Was Autopilot enabled?", while with most other car crashes, it's just a car crash. So I don't think that lack of news about other cars with TACC means that they don't suffer from similar issues. (TBH I wouldn't be able to name any other cars with such an advanced TACC either).
  - shereadsthenews 8y ago
    Tesla’s popularity? It is the twentieth-best-selling brand.
VikingCoder 8y ago
Sounds like TACC is deadly, then. Can Waymo handle this situation or not?
rayiner 8y ago
Tesla's system is not the best: https://www.caranddriver.com/features/a24511826/safety-features-automatic-braking-system-tested-explained https://www.caranddriver.com/features/a24511826/safety-featu....

jameskilton 8y ago

This has been true for a long time. Radar can't differentiate between a wall, a stopped car, or a dead skunk. Heck, it can get confused with a bridge's shadow on a bright day. Thus autopilot has to be configured to deal with these situations: stop or run into them. The vast majority of the situation you don't want the car to stop, as that will lead to a more dangerous situation and increase the chance of a collision. It's one of the many situations that show that "autopilot", particularly Level 4 and Level 5, have a very, very long way to go. Also, as the reddit thread discusses the psychology of autopilot, this is why I don't want self-driving cars yet. Humans are horrendous at paying attention when they aren't directly in control. Airlines have been dealing with this problem for decades. Tesla has even started calling their services "driver assist" instead of autopilot to help combat this problem. And yes, many people use autopilot without a problem every day, but you have to remember that it only takes one single miss to kill you (see: driver killed by ramming into a lane divider on autopilot).

tntn 8y ago
> Radar can't differentiate between a wall, a stopped car, or a dead skunk "Radar" absolutely can. What can't differentiate between these things is the radar in Teslas that was intended for use in adaptive cruise control. There are any number of not insanely expensive radars that could do this job perfectly fine. Tesla just decided to go "full self driving" with this rather bad radar because "humans can do it with vision only!!!1!!!1!"
- Junk_Collector 8y ago
  I don't the specifics of Tesla's radar implementation but what I have seen of the accidents usually seem like a textbook examples of incorrectly implemented clutter filtering. It's not an easy problem to solve in practice (re: cheaply), but known solutions have existed for decades.
  - tntn 8y ago
    The radar module in Teslas uses the same filtering strategy as most adaptive cruise control radar modules, which is "if relative velocity to object is nearly the opposite of the car's velocity, throw it out." It works pretty much as designed.

stevoo 8y ago

This not news. This has existed from the first iteration of Tesla Autopilots. That is why there where crashes with stationery vehicles for Tesla. This is shortcoming for using only cameras or any vehicle that has adaptive cruise control. The problem is that the front camera cannot distinguish from the road and a stationery vehicle. For that, they are the same and they ignore stationary obstacles. If they where taken into consideration, then they would not work at all. This is where Lidar comes in but at its current price, or at price that Tesla was designing its system it was too expensive. I am not entirely sure that pure cameras will ever be enough

derekp7 8y ago
This is something I don't understand. If you have more than 1 camera, you can run an edge detection algorithm on the images from each camera and use simple trig to build up a 3d model of what is in front of the car. This can easily distinguish between a shadow or road kill, vs a large object in the way. Is there a technical reason they don't do this (such as too much processing power required)?
- rcxdude 8y ago
  You are trivialising the problem. Getting reliable data from stereo images is still far from solved, especially in realtime. (Also, the best currently available techniques are likely too computationally expensive to run on the compute that Tesla has put in the car).
  - fabianluque 8y ago
    You can clearly see here (https://www.youtube.com/watch?v=ROHW3sm2j0g https://www.youtube.com/watch?v=ROHW3sm2j0g) that Autopilot is detecting moving, stationary, stopped and oncoming vehicles. There could be situations where this does not work accurately and that's when problems occur. If you watch closely you can even see it sometimes in this video.
    - kevin_thibedeau 8y ago

athenot 8y ago

Yet isn't this addressed by the "dumb" collision avoidance system found in many premium cars? ("dumb" as in limited in scope and not trying to drive the car as a whole.)

gthtjtkt 8y ago

What are we going to find out next, water is wet? Every radar cruise control system comes with a warning that it can't detect stopped traffic. This seems like a case of RTFM.

tntn 8y ago
The issue is that tesla packaged up a radar designed for adaptive cruise control as part of their "autopilot." Maybe there is some disclaimer somewhere that says autopilot will happily run into stopped cars, but I don't know that most drivers of Teslas are aware of this.
- gthtjtkt 8y ago
  It's in the manual. In fact, the exact scenario in this video is on page 83 of the Model S manual.

js2 8y ago

Readers might find this piece (“Four Cars Repeatedly Crashed to Test the Latest Automatic Braking Safety Systems”) interesting: https://www.caranddriver.com/features/a24511826/safety-features-automatic-braking-system-tested-explained/ https://www.caranddriver.com/features/a24511826/safety-featu...

gopher2 8y ago

It's so bad that they call this feature "Autopilot". Taking a page out of the AT&T 5G Evolution playbook I guess.

xnyan 8y ago

A LIDAR array is a requirement for automated driving. It can operate in some conditions that radar cannot. lidar, radar and computer vision should all be used in concert for best effect and Tesla knows this, but chooses/chose not to in order to meet a price point. That choice may have been the right one, I don't know. Hacker news (and me too, to be honest) focuses on Tesla, but there are a lot of other players doing this and doing it better IMO.

benjohnson 8y ago

This is rather useful.. If I notice a Tesla behind me, I'll do my best not to come to an abrupt complete stop. Sounds like if I slow down gradually, it will give it enough data to slow down as well.

kwhitefoot 8y ago

This is not news to anyone who own a Tesla. Perhaps someone who drives a car with a different radar could tell us if any other real world systems (actually installed on mass market vehicles right now) does better.

rayiner 8y ago

Tesla's system is not only not "autopilot" (it will happily plow into stationary vehicles), it's not even the best adaptive cruise control on the market for dealing with stationary vehicles: https://www.caranddriver.com/features/a24511826/safety-features-automatic-braking-system-tested-explained https://www.caranddriver.com/features/a24511826/safety-featu.... In a recent Car & Driver test, Subaru beat Tesla in every single scenario involving fast approach to a stationary/slow-moving target (although all collided with the target at closing speeds >= 50 mph).

Tomminn 8y ago

Is this just because it's hard to distinguish using visual data when it's a 2-D image on the road (from the sun, intentional paint, a spill etc) and when its a 3-D object sitting upright on the road?