47 ms·

In the increasingly rare instances where Tesla's solution is making mistakes, it's pretty much never to do with a failure of spatial awareness (sensing) but rat

In the increasingly rare instances where Tesla's solution is making mistakes, it's pretty much never to do with a failure of spatial awareness (sensing) but rather a failure of path planning (decision-making). The only thing LIDAR can do sense depth, and if it turns out sensing depth using cameras is a solved problem, adding LIDAR doesn't help. It can't read road signs. It can't read road lines. It can't tell if a traffic light is red or green. And it certainly doesn't improve predictions of human drivers.

dzhiurgis 10mo ago
Which begs me the question why Tesla took so long to get here? It's only since v12 it starting to look bearable for supervised use. The only answer I see is their goal to create global model that works in every part of the world vs single city which is vastly more difficult. After all most drivers really only know how to drive well in their own town and make a lot of mistakes when driving somewhere else.
- simondotau 10mo ago
  Path planning (decision-making) is by far the most complicated part of self-driving. Waymo vehicles were making plenty of comically stupid mistakes early on, because having sufficient spatial accuracy was never the truly hard part.
- Rover222 10mo ago
  It was only about 2 years ago that they switched from hard coded logic to machine learning (video in, car control out), and this was the beginning of their final path they are committed to now. (building out manufacturing for Cybercab while still finalizing the FSD software is a pretty insane risk that no other company would take)
  - dzhiurgis 10mo ago
    That’s the switch for controls, the machine vision was nn from the start.
KeplerBoy 10mo ago
Sensing depth is pretty important though. Especially in scenarios where vision fails, radar for example works perfectly fine in the thickest of fog.
- simondotau 10mo ago
  In "scenarios where vision fails" the car should not be driving. Period. End of story. It doesn't matter how good radar is in fog, because radar alone is not enough.
  - KeplerBoy 10mo ago
    Too bad conditions can change instantly. You can't stop the car at an alpine tunnel exit just because there's heavy fog on the other side of the mountain.
    - simondotau 10mo ago
      If the fog is thick enough that you literally can't see the road, you absolutely can and should stop. Most of the time there's still some visibility through fog, and so your speed should be appropriate to the conditions. As the saying goes, "don't drive faster than your headlights."
ra7 10mo ago
> The only thing LIDAR can do sense depth This is absolutely false. LiDAR is used heavily in object detection. There’s plenty of literature on this. Here’s a few from Waymo: https://waymo.com/research/streaming-object-detection-for-3-d-point-clouds/ https://waymo.com/research/streaming-object-detection-for-3-... https://waymo.com/research/lef-late-to-early-temporal-fusion-for-lidar-3d-object-detection/ https://waymo.com/research/lef-late-to-early-temporal-fusion... https://waymo.com/research/3d-human-keypoints-estimation-from-point-clouds-in-the-wild-without-human/ https://waymo.com/research/3d-human-keypoints-estimation-fro... In fact, LiDAR is a key component for detecting pedestrian keypoints and pose estimation. See https://waymo.com/blog/2022/02/utilizing-key-point-and-pose-estimation https://waymo.com/blog/2022/02/utilizing-key-point-and-pose-... Here’s an actual example of LiDAR picking up people in the dark well before cameras: https://www.reddit.com/r/waymo/s/U8eq8BEaGA https://www.reddit.com/r/waymo/s/U8eq8BEaGA Not to mention they’re also highly critical for simulation. > It can't read road signs. It can't read road lines. Also false. Here’s Waymo’s 5th-gen LiDAR raw point clouds that can even read a logo on a semi truck: https://youtube.com/watch?v=COgEQuqTAug&t=11600s https://youtube.com/watch?v=COgEQuqTAug&t=11600s It seems you’re misinformed about how this sensor is used. The point clouds (plus camera and radar data) are all fed to the models for detection. That makes their detectors much more robust in different lighting and weather conditions than cameras alone.
- Rover222 10mo ago
  I think "sensing depth" and "object detection" are the same things in this debate though
  - ra7 10mo ago
    It's just "sensing depth" the same way cameras provide just "pixels". A fused cameras+radars+lidar input provides more robust coverage in a variety of conditions.
    - simondotau 10mo ago
      You know it would be even more robust under even more conditions? Putting 80 cameras and 20 LIDAR sensors on the car. Also a dozen infrared heat sensors, a spectrophotometer, and a Doppler radar. More is surely always better. Waymo should do that.

dzhiurgis 10mo ago

Which begs me the question why Tesla took so long to get here? It's only since v12 it starting to look bearable for supervised use. The only answer I see is their goal to create global model that works in every part of the world vs single city which is vastly more difficult. After all most drivers really only know how to drive well in their own town and make a lot of mistakes when driving somewhere else.

simondotau 10mo ago
Path planning (decision-making) is by far the most complicated part of self-driving. Waymo vehicles were making plenty of comically stupid mistakes early on, because having sufficient spatial accuracy was never the truly hard part.
Rover222 10mo ago
It was only about 2 years ago that they switched from hard coded logic to machine learning (video in, car control out), and this was the beginning of their final path they are committed to now. (building out manufacturing for Cybercab while still finalizing the FSD software is a pretty insane risk that no other company would take)
- dzhiurgis 10mo ago
  That’s the switch for controls, the machine vision was nn from the start.

KeplerBoy 10mo ago

Sensing depth is pretty important though. Especially in scenarios where vision fails, radar for example works perfectly fine in the thickest of fog.

simondotau 10mo ago
In "scenarios where vision fails" the car should not be driving. Period. End of story. It doesn't matter how good radar is in fog, because radar alone is not enough.
- KeplerBoy 10mo ago
  Too bad conditions can change instantly. You can't stop the car at an alpine tunnel exit just because there's heavy fog on the other side of the mountain.
  - simondotau 10mo ago
    If the fog is thick enough that you literally can't see the road, you absolutely can and should stop. Most of the time there's still some visibility through fog, and so your speed should be appropriate to the conditions. As the saying goes, "don't drive faster than your headlights."

ra7 10mo ago

> The only thing LIDAR can do sense depth This is absolutely false. LiDAR is used heavily in object detection. There’s plenty of literature on this. Here’s a few from Waymo: https://waymo.com/research/streaming-object-detection-for-3-d-point-clouds/ https://waymo.com/research/streaming-object-detection-for-3-... https://waymo.com/research/lef-late-to-early-temporal-fusion-for-lidar-3d-object-detection/ https://waymo.com/research/lef-late-to-early-temporal-fusion... https://waymo.com/research/3d-human-keypoints-estimation-from-point-clouds-in-the-wild-without-human/ https://waymo.com/research/3d-human-keypoints-estimation-fro... In fact, LiDAR is a key component for detecting pedestrian keypoints and pose estimation. See https://waymo.com/blog/2022/02/utilizing-key-point-and-pose-estimation https://waymo.com/blog/2022/02/utilizing-key-point-and-pose-... Here’s an actual example of LiDAR picking up people in the dark well before cameras: https://www.reddit.com/r/waymo/s/U8eq8BEaGA https://www.reddit.com/r/waymo/s/U8eq8BEaGA Not to mention they’re also highly critical for simulation. > It can't read road signs. It can't read road lines. Also false. Here’s Waymo’s 5th-gen LiDAR raw point clouds that can even read a logo on a semi truck: https://youtube.com/watch?v=COgEQuqTAug&t=11600s https://youtube.com/watch?v=COgEQuqTAug&t=11600s It seems you’re misinformed about how this sensor is used. The point clouds (plus camera and radar data) are all fed to the models for detection. That makes their detectors much more robust in different lighting and weather conditions than cameras alone.

Rover222 10mo ago
I think "sensing depth" and "object detection" are the same things in this debate though
- ra7 10mo ago
  It's just "sensing depth" the same way cameras provide just "pixels". A fused cameras+radars+lidar input provides more robust coverage in a variety of conditions.
  - simondotau 10mo ago
    You know it would be even more robust under even more conditions? Putting 80 cameras and 20 LIDAR sensors on the car. Also a dozen infrared heat sensors, a spectrophotometer, and a Doppler radar. More is surely always better. Waymo should do that.