3 ms·

The lesson from Tesla is that AI is not just a magic box where you can put in data and get out intelligence. There are more to working systems than compute, and

The lesson from Tesla is that AI is not just a magic box where you can put in data and get out intelligence. There are more to working systems than compute, and when they operate in the real world, data isn't enough. The key problem with Tesla cars that keep them from succeeding is not that they don't have enough data, but they have no idea what to do with it. Even if they had infinite compute and all the driving videos in the world, it wouldn't be enough to overcome the limitations of their sensors.

dangus 2y ago
Tesla is a poor counterargument because it is no longer a market leader. It has poor management compared to 10 years ago and seems to be unable to attract top talent (poor labor relations). Tesla is being leapfrogged by competitors across the auto industry. All it has is first mover status (charging network). Tesla purposefully limits the capabilities of its self driving by refusing to implement it with sensors that go beyond smartphone cameras. My belief is that Tesla doesn’t want to actually deliver a car that can drive itself because the end result of Waymo is that fewer people will need to own a car and fleets of short term rental self-driving cars won’t spend frivolous money on prestige and luxury like consumer car buyers. They won’t lease a car and replace it every 2-3 years like some car owners do just because they like having a new car. Fleet vehicle operators purchase cars with razor thin margins and make decisions based solely on economics, as well as having a lot more purchasing leverage over car manufacturers. I don’t think Tesla ever wants self driving to work, they just want to sell the idea of the software.
- immibis 2y ago
  Tesla removed the LIDAR and thought advances in AI would be able to do without one. They were wrong.
  - _ea1k 2y ago
    Tesla didn't remove LIDAR, they never had it. So far, that bet is looking pretty reasonable. It seems evident at the moment that the most formidable competitors in this space could build a solid FSD product with cameras alone, with the biggest variable being time.
    - dangus 2y ago
      Is that evident? The only level 3 certified drive system available in the US (Mercedes) utilizes LIDAR. The only operating robotaxi service also uses it. I would say it’s the opposite of evident that cameras are enough.
      - _ea1k 2y ago
        While Mobileye has focused on using lidar+cameras with each effectively serving as a form of "backup", they have also demonstrated a full city drive using nothing but cameras. I've seen enough from Waymo to see that they could easily do the same thing. I expect both to keep radar and lidar for quite a few years, but I think its usage will consistently decline to zero or near zero over the next 10 years.
        ModernMech 2y ago
        The problem with this prediction is.... why? Camera doesn't actually replace LiDAR, even if we solve getting depth maps from camera 100%. The recent test in which the car ran over a child hidden in a smoke screen is evidence of this. If they purpose of driverless cars is to be better than humans, we should be aiming for them to have super human sensing with camera+lidar+radar etc, instead of human-level sensing with just cameras.
        _ea1k 2y ago
        Sure, but that test was tailor made for lidar to "succeed" even if lidar actually couldn't see the hidden obstacle. The lidar was largely blocked by the "rain", which meant it could treat it like a wall. For a real world example of this, see Waymo's experience with a water main break: https://www.reddit.com/r/SelfDrivingCars/comments/1g75ftb/waymo_meets_water_fountain/ https://www.reddit.com/r/SelfDrivingCars/comments/1g75ftb/wa... The lidar didn't necessarily see the child, but it didn't have to. A real test of lidar would have been perfectly transparent obstacles. It wouldn't have been realistic, but neither were the other tests. There are two things that will drive companies to eliminate it: - Cost - Even though the cost can come down quite a bit, the current multi-lidar systems they add both up front cost and maintenance cost. - Complexity - They don't do the same things that cameras do, and can in fact never replace cameras. This means that they are not 100% fidelity backups, and therefore the decision of which sensor to be used needs to be made. A Mobileye like solution is likely the best answer here, with a discriminator network to (hopefully) accurately judge which sensor to believe for each detected point, but this is not trivial. In fact, just getting this right may be nearly as hard as the driving policy tasks themselves. I still think lidar may stay around for special purpose tasks (eg, curb detection, or low obstacles), but the era of them being relied on heavily to detect VRUs or other vehicles will surely end.
xg15 2y ago
> The key problem with Tesla cars that keep them from succeeding is not that they don't have enough data, but they have no idea what to do with it. Even if they had infinite compute and all the driving videos in the world, it wouldn't be enough to overcome the limitations of their sensors. Isn't this effectively a refutation of the "bitter lesson"?

dangus 2y ago

Tesla is a poor counterargument because it is no longer a market leader. It has poor management compared to 10 years ago and seems to be unable to attract top talent (poor labor relations). Tesla is being leapfrogged by competitors across the auto industry. All it has is first mover status (charging network). Tesla purposefully limits the capabilities of its self driving by refusing to implement it with sensors that go beyond smartphone cameras. My belief is that Tesla doesn’t want to actually deliver a car that can drive itself because the end result of Waymo is that fewer people will need to own a car and fleets of short term rental self-driving cars won’t spend frivolous money on prestige and luxury like consumer car buyers. They won’t lease a car and replace it every 2-3 years like some car owners do just because they like having a new car. Fleet vehicle operators purchase cars with razor thin margins and make decisions based solely on economics, as well as having a lot more purchasing leverage over car manufacturers. I don’t think Tesla ever wants self driving to work, they just want to sell the idea of the software.

immibis 2y ago
Tesla removed the LIDAR and thought advances in AI would be able to do without one. They were wrong.
- _ea1k 2y ago
  Tesla didn't remove LIDAR, they never had it. So far, that bet is looking pretty reasonable. It seems evident at the moment that the most formidable competitors in this space could build a solid FSD product with cameras alone, with the biggest variable being time.
  - dangus 2y ago
    Is that evident? The only level 3 certified drive system available in the US (Mercedes) utilizes LIDAR. The only operating robotaxi service also uses it. I would say it’s the opposite of evident that cameras are enough.
    - _ea1k 2y ago
      While Mobileye has focused on using lidar+cameras with each effectively serving as a form of "backup", they have also demonstrated a full city drive using nothing but cameras. I've seen enough from Waymo to see that they could easily do the same thing. I expect both to keep radar and lidar for quite a few years, but I think its usage will consistently decline to zero or near zero over the next 10 years.
      - ModernMech 2y ago
        The problem with this prediction is.... why? Camera doesn't actually replace LiDAR, even if we solve getting depth maps from camera 100%. The recent test in which the car ran over a child hidden in a smoke screen is evidence of this. If they purpose of driverless cars is to be better than humans, we should be aiming for them to have super human sensing with camera+lidar+radar etc, instead of human-level sensing with just cameras.
        _ea1k 2y ago
        Sure, but that test was tailor made for lidar to "succeed" even if lidar actually couldn't see the hidden obstacle. The lidar was largely blocked by the "rain", which meant it could treat it like a wall. For a real world example of this, see Waymo's experience with a water main break: https://www.reddit.com/r/SelfDrivingCars/comments/1g75ftb/waymo_meets_water_fountain/ https://www.reddit.com/r/SelfDrivingCars/comments/1g75ftb/wa... The lidar didn't necessarily see the child, but it didn't have to. A real test of lidar would have been perfectly transparent obstacles. It wouldn't have been realistic, but neither were the other tests. There are two things that will drive companies to eliminate it: - Cost - Even though the cost can come down quite a bit, the current multi-lidar systems they add both up front cost and maintenance cost. - Complexity - They don't do the same things that cameras do, and can in fact never replace cameras. This means that they are not 100% fidelity backups, and therefore the decision of which sensor to be used needs to be made. A Mobileye like solution is likely the best answer here, with a discriminator network to (hopefully) accurately judge which sensor to believe for each detected point, but this is not trivial. In fact, just getting this right may be nearly as hard as the driving policy tasks themselves. I still think lidar may stay around for special purpose tasks (eg, curb detection, or low obstacles), but the era of them being relied on heavily to detect VRUs or other vehicles will surely end.

xg15 2y ago

> The key problem with Tesla cars that keep them from succeeding is not that they don't have enough data, but they have no idea what to do with it. Even if they had infinite compute and all the driving videos in the world, it wouldn't be enough to overcome the limitations of their sensors. Isn't this effectively a refutation of the "bitter lesson"?