4 ms·

Which sensor has primacy? radar, lidar or camera? I recall it mentioned somewhere that, after a well publicized tragedy hitting the side of a white truck on a

Which sensor has primacy? radar, lidar or camera? I recall it mentioned somewhere that, after a well publicized tragedy hitting the side of a white truck on a bright day, Tesla was moving to radar? And yet a lot of the cues this awesome video shows seems to be camera-based line detection etc?

jrockway 7y ago
If you look at the overlay, the algorithm seems extra paranoid about the following aspects: HIGH_BEAM, BLINDED, RAINING, TIRE_SPRAY, which all sound like camera weaknesses, not lidar/radar weaknesses. My guess is that this is camera-based still. Also interesting is the vision fps of ~13. That's 75ms latency.
- yathern 7y ago
  75ms sounds like a lot, but it isn't so bad - average human reaction time is about 200ms. Source: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4456887/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4456887/
pixelHD 7y ago
IIRC tesla still doesn't use LIDAR. Just wanted to point that out.
- gandhi00 7y ago
  Correct. I believe they use vision along with front facing radar only
deleted 7y ago
[deleted]
modeless 7y ago
Tesla doesn't use lidar. They have always used radar, but radar can't always tell the difference between a stationary metal sign near/over the road and a stalled car in the middle of the lane, so for stationary objects they need the cameras. This is why every adaptive cruise control system (not just Tesla's) has a warning that it may not detect stationary objects. https://arstechnica.com/cars/2018/06/why-emergency-braking-systems-sometimes-hit-parked-cars-and-lane-dividers/ https://arstechnica.com/cars/2018/06/why-emergency-braking-s...
- tjoff 7y ago
  Though only(?) Tesla pretends that their adaptive cruise control is full blown self-driving.
  - Erlich_Bachman 7y ago
    Source? If you simply refer to it being called "autopilot", cmon that's not too serious. The word "autopilot" does not automatically mean that it is full self-driving. None of the Tesla drivers or potential Tesla buyers realistically think that it has level 5 self-driving yet. UPD: The buy page literally says: "The currently enabled features require active driver supervision and do not make the vehicle autonomous. The activation and use of these features are dependent on achieving reliability far in excess of human drivers as demonstrated by billions of miles of experience, as well as regulatory approval, which may take longer in some jurisdictions. As these self-driving features evolve, your car will be continuously upgraded through over-the-air software updates."
    - aguyfromnb 7y ago
      >Source? If you simply refer to it being called "autopilot", cmon that's not too serious The Model 3 Builder literally says: "Full Self-Driving Capability".
      - Erlich_Bachman 7y ago
        Did you just read the one sentence and not any other text on that page?: "Coming later this year: Automatic driving on city streets.", Oh and how about "The currently enabled features require active driver supervision and do not make the vehicle autonomous. The activation and use of these features are dependent on achieving reliability far in excess of human drivers as demonstrated by billions of miles of experience, as well as regulatory approval, which may take longer in some jurisdictions. As these self-driving features evolve, your car will be continuously upgraded through over-the-air software updates.". - Which I just copy-pasted from the exact page you are talking about. Who is being disingenious here? I honestly don't think that there is any significant number of potential buyers who go onto a website to buy a car, and pull one sentence from that page out of context, ignore the rest of the text and then make their decision based on that.
mitchellgoffpc 7y ago
They use both radar and cameras, but primarily cameras. Lines, road markings, stop lights, and street signs can only be detected with vision, but cars+pedestrians can (usually) be verified by both radar and cameras. In fact, they’ve mentioned that they use radar data to help train the camera neural nets to predict distances more accurately. However, radar alone isn’t sufficient for detecting obstacles — it sometimes misses vehicles with a lot of ground clearance, and sometimes registers false positives when driving underneath bridges and overpasses, resulting in phantom braking.
Symmetry 7y ago
The problem with that collision is that neither the radar nor the camera saw the obstacle. To the camera it looked too much like the sky as you say. To the radar it appeared to be a stationary object and so probably an overhead sign or something instead of another vehicle. Once the car gets close enough that the radar's cone only includes dangerous obstacles then it starts putting the brakes but in that case it was too late. In general, though, the art of combining information from multiple sensors is called sensor fusion and its a fairly well understood problem. Well, sometimes even good engineers make mistakes, see Schiaparelli and the saturated Kalman filter inputs, but we have a good idea of what those mistakes can be for different schema. EDIT: Mistyped radar as lidar, fixed.
- threeseed 7y ago
  Tesla doesn't use LiDAR. And I assure you that sensor fusion is not a well understood problem. One of the biggest challenges in autonomous driving is how to unify all of the different NN outputs.

jrockway 7y ago

If you look at the overlay, the algorithm seems extra paranoid about the following aspects: HIGH_BEAM, BLINDED, RAINING, TIRE_SPRAY, which all sound like camera weaknesses, not lidar/radar weaknesses. My guess is that this is camera-based still. Also interesting is the vision fps of ~13. That's 75ms latency.

yathern 7y ago
75ms sounds like a lot, but it isn't so bad - average human reaction time is about 200ms. Source: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4456887/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4456887/

pixelHD 7y ago

IIRC tesla still doesn't use LIDAR. Just wanted to point that out.

gandhi00 7y ago
Correct. I believe they use vision along with front facing radar only

deleted 7y ago

[deleted]

modeless 7y ago

Tesla doesn't use lidar. They have always used radar, but radar can't always tell the difference between a stationary metal sign near/over the road and a stalled car in the middle of the lane, so for stationary objects they need the cameras. This is why every adaptive cruise control system (not just Tesla's) has a warning that it may not detect stationary objects. https://arstechnica.com/cars/2018/06/why-emergency-braking-systems-sometimes-hit-parked-cars-and-lane-dividers/ https://arstechnica.com/cars/2018/06/why-emergency-braking-s...

tjoff 7y ago
Though only(?) Tesla pretends that their adaptive cruise control is full blown self-driving.
- Erlich_Bachman 7y ago
  Source? If you simply refer to it being called "autopilot", cmon that's not too serious. The word "autopilot" does not automatically mean that it is full self-driving. None of the Tesla drivers or potential Tesla buyers realistically think that it has level 5 self-driving yet. UPD: The buy page literally says: "The currently enabled features require active driver supervision and do not make the vehicle autonomous. The activation and use of these features are dependent on achieving reliability far in excess of human drivers as demonstrated by billions of miles of experience, as well as regulatory approval, which may take longer in some jurisdictions. As these self-driving features evolve, your car will be continuously upgraded through over-the-air software updates."
  - aguyfromnb 7y ago
    >Source? If you simply refer to it being called "autopilot", cmon that's not too serious The Model 3 Builder literally says: "Full Self-Driving Capability".
    - Erlich_Bachman 7y ago
      Did you just read the one sentence and not any other text on that page?: "Coming later this year: Automatic driving on city streets.", Oh and how about "The currently enabled features require active driver supervision and do not make the vehicle autonomous. The activation and use of these features are dependent on achieving reliability far in excess of human drivers as demonstrated by billions of miles of experience, as well as regulatory approval, which may take longer in some jurisdictions. As these self-driving features evolve, your car will be continuously upgraded through over-the-air software updates.". - Which I just copy-pasted from the exact page you are talking about. Who is being disingenious here? I honestly don't think that there is any significant number of potential buyers who go onto a website to buy a car, and pull one sentence from that page out of context, ignore the rest of the text and then make their decision based on that.

mitchellgoffpc 7y ago

They use both radar and cameras, but primarily cameras. Lines, road markings, stop lights, and street signs can only be detected with vision, but cars+pedestrians can (usually) be verified by both radar and cameras. In fact, they’ve mentioned that they use radar data to help train the camera neural nets to predict distances more accurately. However, radar alone isn’t sufficient for detecting obstacles — it sometimes misses vehicles with a lot of ground clearance, and sometimes registers false positives when driving underneath bridges and overpasses, resulting in phantom braking.

Symmetry 7y ago

The problem with that collision is that neither the radar nor the camera saw the obstacle. To the camera it looked too much like the sky as you say. To the radar it appeared to be a stationary object and so probably an overhead sign or something instead of another vehicle. Once the car gets close enough that the radar's cone only includes dangerous obstacles then it starts putting the brakes but in that case it was too late. In general, though, the art of combining information from multiple sensors is called sensor fusion and its a fairly well understood problem. Well, sometimes even good engineers make mistakes, see Schiaparelli and the saturated Kalman filter inputs, but we have a good idea of what those mistakes can be for different schema. EDIT: Mistyped radar as lidar, fixed.

threeseed 7y ago
Tesla doesn't use LiDAR. And I assure you that sensor fusion is not a well understood problem. One of the biggest challenges in autonomous driving is how to unify all of the different NN outputs.