4 ms·

Ever since iPhone introduced Lidar, I've wondered why Teslas got rid of it. They said it was due to additional cost and complexity in models. Of course Waymo he

Ever since iPhone introduced Lidar, I've wondered why Teslas got rid of it. They said it was due to additional cost and complexity in models. Of course Waymo heavily uses Lidar but they have huge sensors they mount on cars to do so. Why is that lidar so different? What's stopping Tesla from adding the same sensors the iphone has, but all over the car and stitching it together like it does video from cameras? Is it just to do with distance? Refresh rate? Would love someone who knows this to teach me!

SomeHacker44 4y ago
AFAIK Tesla never had lidar. They had and will again have simple radar.
Melatonic 4y ago
Lidar I believe can be heavily affected by weather and might also require a lot of expensive sensors. It works great for VFX work when you need a precise model of an area. But I would guess the refresh rate is also not amazing. I live near Waymo and see the cars all the time - they do have huge things on top but the weather in Santa Monica is also very good. Not sure if I have seen them ever driving in the rain?
- zxexz 4y ago
  Interesting. I’ve used a Velodyne VLP-32 in reasonably heavy rain with reasonable performance after some error correction. I was on foot and not driving, and probably only for a few hours total. But it seems to me that a big company with serious engineering talent should be able to make it work.
  - throwbadubadu 4y ago
    Yes.. any other car manufacturer I know of (likely not complete) is considering also Lidar sensors... they are still pricey and huge, but there are also MEMS-like improvements (likely incorrect terminology) and also costs get continously lower. At the same time advancements also happen at the multi-model side, not fusing different deep learning outputs afterwards but right in the one model (not judging if that is a good idea... but the steps taking currently seem to get bigger and better). Lastly, also camera has degraded performance under certain weather conditions.. some even say our eyes can be prone to that! :D So that's just all the challenges that need to be taken. Yes, humans can drive with their eyes only, but is that perfect? Sometimes we e.g. would also benefit greatly from radar/lidar, as e.g. in dark conditions our judgement of distances is really flawed. Refresh rate of some of this already aged car lidars was like 10 Hz, good enough I'd say? And it is truly amazing what details one can get out of those when getting close and move around in those point clouds.
    - LeanderK 4y ago
      It's been a bit since I've been out of touch with the autonomous driving space. Was there significant progress the last 3 years? What are the current challenges?
recuter 4y ago
Simply because they are all in on cameras although there are recent teardowns of their cars online that foreshadow a possible comeback for additional sensors.
bri3d 4y ago
There are a lot of different things called "Lidar" : at a minimum, Structured Light, Time of Flight, and triangulation are the three most common methods. Within each method, there are vastly different approaches - scanning vs. single-frame, frequency modulated vs. pulsed, and more. Each approach comes with different trade-offs suited to different use-cases. In the case of the iPhone rear camera, I believe the approach used is single-frame pulsed Lidar using a VCSEL (vertical cavity surface-emitting laser), a dot-grid diffuser, and SPAD (single photon avalanche diode) CMOS camera. Basically, a shifting grid of dots at a very specific wavelength is projected outwards from the laser, and an unbelievably, photon-level sensitive camera tuned to that precise wavelength measures how long each dot takes to come back after it's emitted. In this way, Time of Flight can be measured and a precise distance determined. This system provides incredibly high resolution at a moderate distance (in this case, 5m or so) at a moderate cost. For a car, this system wouldn't be very plausible due to range, resolution, and output power constraints. The size of laser required to emit a coherent dot matrix which could reflect at a long distance would grow to an extreme degree, the noise floor of the SPAD would eventually render the system useless, and the amount of data required to produce a higher resolution dot cloud would become quite challenging, as these systems generally need to be heavily filtered and sub-sampled, so an enormous amount of input data processing is required to produce a reasonable output. The main reason that Tesla don't use Lidar is that they decided to bet on optical (IMO, foolish), but the reason they decided to bet on optical is complex and definitely in no small way cost driven. They couldn't just slap an array of iPhone sensors (I believe they're Sony-manufactured) on a Tesla and call it good.
ororroro 4y ago
Tesla did not add iPhone style lidar (up to 5m) because they already had 12 cheap ultrasonic sensors (up to 8m) for short range distance detection since all they need to know at that range is will or won't I hit something. They are now full optical though and may have already been ignoring some sensors in software. https://en.wikipedia.org/wiki/Tesla_Autopilot#Hardware https://en.wikipedia.org/wiki/Tesla_Autopilot#Hardware There are several reasons why they would not just add new sensors even if they are in some/all ways superior: * Tesla special cases different driving situations. This makes sense as a product company seeking to sell consumers on solutions to each situation as a feature but means they need to re-implement each situation individually when the sensors change rather than just swapping out some generic distance detector net. You can observe this from the loss of features when Tesla went full Vision over the last two years. * Adding new sensors adds new errors which leaves you to chose between ignoring danger conditions present in only one sensor as possible errors or greatly increasing safety stops etc (some of which would be genuine previously undetected dangers). Because of Tesla's engineering culture the prudent second option or a sensible middle ground would be impossible to make since they would fail from a user experience point of view at the first false stop during a demo to the big man and then be forever poisoned (imho). * Tesla has always been pursuing a full vision only solution on the reasonable assumption that if humans can do it then eventually computers will too. The only disagreement with that strategy is when exactly that will/should be.
- BoorishBears 4y ago
  I like how you ignored the truest reason: they don't want to pay for it, and they don't really care if the current hardware can actually solve FSD. There's no way you can argue it was done in good faith when it was an arbitrary transition where some functionality hadn't even been solved yet and was disabled if your car didn't come with ultrasonic sensors. And to add to the comical levels of disregard for buyers, the official advice for finding out if your order would have ultrasonic sensors was: "Look for the 12 sensors on your vehicle", ie you didn't know until you paid and it was a roulette wheel.
  - ororroro 4y ago
    I don't think the price of a $10 sensor would trump Elon's desire to be right about vision. Price was a factor for not using the larger, longer range lidars though since that cost more than planned price of the vehicle ~10 years ago.
latchkey 4y ago
https://www.forbes.com/sites/bradtempleton/2022/10/31/former-head-of-tesla-ai-explains-why-theyve-removed-sensors-others-differ/ https://www.forbes.com/sites/bradtempleton/2022/10/31/former...

SomeHacker44 4y ago

AFAIK Tesla never had lidar. They had and will again have simple radar.

Melatonic 4y ago

Lidar I believe can be heavily affected by weather and might also require a lot of expensive sensors. It works great for VFX work when you need a precise model of an area. But I would guess the refresh rate is also not amazing. I live near Waymo and see the cars all the time - they do have huge things on top but the weather in Santa Monica is also very good. Not sure if I have seen them ever driving in the rain?

zxexz 4y ago
Interesting. I’ve used a Velodyne VLP-32 in reasonably heavy rain with reasonable performance after some error correction. I was on foot and not driving, and probably only for a few hours total. But it seems to me that a big company with serious engineering talent should be able to make it work.
- throwbadubadu 4y ago
  Yes.. any other car manufacturer I know of (likely not complete) is considering also Lidar sensors... they are still pricey and huge, but there are also MEMS-like improvements (likely incorrect terminology) and also costs get continously lower. At the same time advancements also happen at the multi-model side, not fusing different deep learning outputs afterwards but right in the one model (not judging if that is a good idea... but the steps taking currently seem to get bigger and better). Lastly, also camera has degraded performance under certain weather conditions.. some even say our eyes can be prone to that! :D So that's just all the challenges that need to be taken. Yes, humans can drive with their eyes only, but is that perfect? Sometimes we e.g. would also benefit greatly from radar/lidar, as e.g. in dark conditions our judgement of distances is really flawed. Refresh rate of some of this already aged car lidars was like 10 Hz, good enough I'd say? And it is truly amazing what details one can get out of those when getting close and move around in those point clouds.
  - LeanderK 4y ago
    It's been a bit since I've been out of touch with the autonomous driving space. Was there significant progress the last 3 years? What are the current challenges?

recuter 4y ago

Simply because they are all in on cameras although there are recent teardowns of their cars online that foreshadow a possible comeback for additional sensors.

bri3d 4y ago

There are a lot of different things called "Lidar" : at a minimum, Structured Light, Time of Flight, and triangulation are the three most common methods. Within each method, there are vastly different approaches - scanning vs. single-frame, frequency modulated vs. pulsed, and more. Each approach comes with different trade-offs suited to different use-cases. In the case of the iPhone rear camera, I believe the approach used is single-frame pulsed Lidar using a VCSEL (vertical cavity surface-emitting laser), a dot-grid diffuser, and SPAD (single photon avalanche diode) CMOS camera. Basically, a shifting grid of dots at a very specific wavelength is projected outwards from the laser, and an unbelievably, photon-level sensitive camera tuned to that precise wavelength measures how long each dot takes to come back after it's emitted. In this way, Time of Flight can be measured and a precise distance determined. This system provides incredibly high resolution at a moderate distance (in this case, 5m or so) at a moderate cost. For a car, this system wouldn't be very plausible due to range, resolution, and output power constraints. The size of laser required to emit a coherent dot matrix which could reflect at a long distance would grow to an extreme degree, the noise floor of the SPAD would eventually render the system useless, and the amount of data required to produce a higher resolution dot cloud would become quite challenging, as these systems generally need to be heavily filtered and sub-sampled, so an enormous amount of input data processing is required to produce a reasonable output. The main reason that Tesla don't use Lidar is that they decided to bet on optical (IMO, foolish), but the reason they decided to bet on optical is complex and definitely in no small way cost driven. They couldn't just slap an array of iPhone sensors (I believe they're Sony-manufactured) on a Tesla and call it good.

ororroro 4y ago

Tesla did not add iPhone style lidar (up to 5m) because they already had 12 cheap ultrasonic sensors (up to 8m) for short range distance detection since all they need to know at that range is will or won't I hit something. They are now full optical though and may have already been ignoring some sensors in software. https://en.wikipedia.org/wiki/Tesla_Autopilot#Hardware https://en.wikipedia.org/wiki/Tesla_Autopilot#Hardware There are several reasons why they would not just add new sensors even if they are in some/all ways superior: * Tesla special cases different driving situations. This makes sense as a product company seeking to sell consumers on solutions to each situation as a feature but means they need to re-implement each situation individually when the sensors change rather than just swapping out some generic distance detector net. You can observe this from the loss of features when Tesla went full Vision over the last two years. * Adding new sensors adds new errors which leaves you to chose between ignoring danger conditions present in only one sensor as possible errors or greatly increasing safety stops etc (some of which would be genuine previously undetected dangers). Because of Tesla's engineering culture the prudent second option or a sensible middle ground would be impossible to make since they would fail from a user experience point of view at the first false stop during a demo to the big man and then be forever poisoned (imho). * Tesla has always been pursuing a full vision only solution on the reasonable assumption that if humans can do it then eventually computers will too. The only disagreement with that strategy is when exactly that will/should be.

BoorishBears 4y ago
I like how you ignored the truest reason: they don't want to pay for it, and they don't really care if the current hardware can actually solve FSD. There's no way you can argue it was done in good faith when it was an arbitrary transition where some functionality hadn't even been solved yet and was disabled if your car didn't come with ultrasonic sensors. And to add to the comical levels of disregard for buyers, the official advice for finding out if your order would have ultrasonic sensors was: "Look for the 12 sensors on your vehicle", ie you didn't know until you paid and it was a roulette wheel.
- ororroro 4y ago
  I don't think the price of a $10 sensor would trump Elon's desire to be right about vision. Price was a factor for not using the larger, longer range lidars though since that cost more than planned price of the vehicle ~10 years ago.

latchkey 4y ago

https://www.forbes.com/sites/bradtempleton/2022/10/31/former-head-of-tesla-ai-explains-why-theyve-removed-sensors-others-differ/ https://www.forbes.com/sites/bradtempleton/2022/10/31/former...