5 ms·

Engineering reliability is primarily achieved through redundancy. There is none with Musk's "vision only" approach. Vision can fail for a multitude of reasons

Engineering reliability is primarily achieved through redundancy. There is none with Musk's "vision only" approach. Vision can fail for a multitude of reasons --- sunlight, rain, darkness, bad road markers, even glare from a dirty windshield. And when it fails, there is no backup plan -- the car is effectively driving blind. Driving is a dynamic activity that involves a lot more than just vision. Safe automated driving can use all the help it can get.

Someone1234 1y ago
I agree with everything you're saying; but even outside of Tesla, I'd just like to remind people that LIDAR as a complement to vision isn't at all straightforward. Sensor fusion adds real complexity in calibration, time sync, and modeling. Both LIDAR and vision have edge cases where they fail. So you ideally want both, but then the challenge is reconciling disagreements with calibrated, and probabilistic fusion. People seem to be under the mistaken impression that vision is dirty input and LIDAR is somehow clean, when in reality both are noisy inputs with different strengths and weaknesses. I guess my point is: Yes, 100% bring in LIDAR, I believe the future is LIDAR + vision. But when you do that, early iterations can regress significantly from vision-only until the fusion is tuned and calibration is tight, because you have to resolve contradictory data. Ultimately the payoff is higher robustness in exchange for more R&D and development workload (i.e. more cost). The same reason why Tesla needed vision-only to work (cost & timeline) is the same reason why vision+LIDAR is so challenging.
- jqpabc123 1y ago
  The same reason why Tesla needed vision-only to work (cost & timeline) But vision only hasn't worked --- not as promised, not after a decade's worth of timeline. And it probably won't any time soon either --- for valid engineering reasons. Engineering 101 --- *needing* something to work doesn't make it possible or practical.
- ethbr1 1y ago
  The primary benefit of multiple sensor fusion from a safety standpoint isn't an absolute decrease in errors. It's the ability to detect sensor disagreements at all. With single modality sensors, you have no way of truly detecting failures in that modality, other than hacks like time-series normalizing (aka expected scenarios). If multiple sensor modalities disagree, even without sensor fusion, you can at least assume something might be awry and drop into a maximum safety operation mode. But we'd think that the budget config of the Boeing 737 MAX would have taught us that tying safety critical systems to single sources of truth is a bad idea... (in that case, critical modality / single physical sensor)
  - leoc 1y ago
    > If multiple sensor modalities disagree, even without sensor fusion, you can at least assume something might be awry and drop into a maximum safety operation mode. Also, this is probably when Waymo calls up a human assistant in a developing-country callcentre.
    - ethbr1 1y ago
      Saw that happen a week ago, actually. Non-sensor problem, but a Waymo made a slow right turn too wide, approached the left turning lane of cars, then safed itself by stopping, then remote assistance came online and extricated it.
  - AnIrishDuck 1y ago
    > With single modality sensors, you have no way of truly detecting failures in that modality, other than hacks like time-series normalizing (aka expected scenarios). "A man with a watch always knows what time it is. If he gains another, he is never sure" Most safety critical systems actually need at least three redundant sensors. Two is kinda useless: if they disagree, which is right? EDIT: > If multiple sensor modalities disagree, even without sensor fusion, you can at least assume something might be awry and drop into a maximum safety operation mode. This is not always possible. You're on a two lane road. Your vision system tells you there's a pedestrian in your lane. Your LIDAR says the pedestrian is actually in the other lane. There's enough time for a lane change, but not to stop. What do you do?
- ra7 1y ago
  The complexity argument rings hollow to me. It’s a bit like saying distributed databases are complex because you have to deal with CAP guarantees. Yes, but people still develop them because it has real benefits. It was maybe a valid argument 10 years ago, but in 2025 many companies have shown sensor fusion works just fine. I mean, Waymo has clocked 100M+ miles, so it works. The AV industry has moved on to more interesting problems, while Tesla and Musk are still stuck in the past arguing about sensor choices.
  - leoc 1y ago
    Well, it's more like sensor fusion plus extensive human remote intervention, it seems: https://www.nytimes.com/interactive/2024/09/03/technology/zoox-self-driving-cars-remote-control.html https://www.nytimes.com/interactive/2024/09/03/technology/zo... . Mind you, if it takes both LiDAR and call-centre workers to make self-driving work in 2025 and for the foreseeable future, that makes Tesla's old ambition to achieve it with neither look all the more hopeless.
    - ra7 1y ago
      Tesla robotaxis have teleoperation [1], which is worse than remote assistance others use because the operators have direct control. Yet they cannot fully remove safety personnel from the car. The old ambition is dead. [1] https://electrek.co/2025/05/16/tesla-robotaxi-fleet-powered-by-plenty-of-teleoperation/ https://electrek.co/2025/05/16/tesla-robotaxi-fleet-powered-...
    - Narciss 1y ago
      V interesting, thanks for sharing, I didn’t know this
- Earw0rm 1y ago
  There's no particular reason to use RGB for this kind of machine vision - cognition problem either. Infra-red of a few different wavelengths as well as optical light ranges seems like it'd give a superior result?
  - jqpabc123 1y ago
    You've just described some of the rationale for using LIDAR.
- maxlin 1y ago
  I think you hit the nail on the head - Obviously when Tesla have saturated the potential of vision, they should bring in LiDAR if it can be reasonably added from a hardware point of view. Their current arguments make this clear - it would be surface-level thinking to add LiDAR and the kitchen sink now, complicating the system's evolution and axing scalability. But we're far from plateauing on what can be done with vision - Humans can drive quite well with essentially just sight, so we're far from extinguishing what can be done with it.
- baby 1y ago
  Sure but if you see something in front of you but LIDAR says "nope I can see 500m away" then you know LIDAR is right
- atcon 1y ago
  Your comments on sensor fusion seem to describe the weird results of 2 informal ADAS (lidar, vision, lidar + vision, lidar + vision + 4d imaging radar, etc.) “tournaments” conducted earlier this year. There was an earlier HN post about it <https://news.ycombinator.com/item?id=44694891 https://news.ycombinator.com/item?id=44694891> with a comment noting “there was a wide range of crash avoidance behavior even between the same car likely due to the machine learning, and that also makes explaining the differences hard. Hopefully someone with more background on ADAS systems can watch and post what they think.” Notably, sensor confusion is also an “unsolved” problem in humans, eg vision and vestibular (inner ear) conflicts possibly explaining motion sickness/vertigo <https://www.nature.com/articles/s44172-025-00417-2 https://www.nature.com/articles/s44172-025-00417-2> The results of both tournaments: <https://carnewschina.com/2025/07/24/chinas-massive-adas-test-36-cars-15-hazard-scenarios-216-crashes/ https://carnewschina.com/2025/07/24/chinas-massive-adas-test...> Counterintuitively, vision scored best (Tesla Model X) The videos are fascinating to watch (subtitles are available): Tournament 1 (36 cars, 6 Highway Scenarios): <https://www.youtube.com/watch?v=0xumyEf-WRI https://www.youtube.com/watch?v=0xumyEf-WRI> Tournament 2 (26 cars, 9 Urban Scenarios): <https://www.youtube.com/watch?v=GcJnNbm-jUI https://www.youtube.com/watch?v=GcJnNbm-jUI> Highway Scenarios: “tests...included other active vehicles nearby to increase complexity and realism”: <https://electrek.co/2025/07/26/a-chinese-real-world-self-driving-test-36-cars-216-crashes-with-tesla-on-top/ https://electrek.co/2025/07/26/a-chinese-real-world-self-dri...> Urban Scenarios: “a massive, complex roundabout and another segment of road with a few unsignaled intersections and a long straight...The first four tests incorporated portions of this huge roundabout, which would be complex for human drivers, but in situations for which there is quite an obvious solution: don’t hit that car/pedestrian in front of you” <https://electrek.co/2025/07/29/another-huge-chinese-self-driving-test-lawbreaking-indecision-in-urban-driving/ https://electrek.co/2025/07/29/another-huge-chinese-self-dri...>
- microtherion 1y ago
  > but then the challenge is reconciling disagreements with calibrated, and probabilistic fusion I keep reading arguments like this, but I really don't understand what the problem here is supposed to be. Yes, in a rule based system, this is a challenge, but in an end-to-end neural network, another sensor is just another input, regardless of whether it's another camera, LIDAR, or a sensor measuring the adrenaline level of the driver. If you have enough training data, the model training will converge to a reasonable set of weights for various scenarios. In fact, training data with a richer set of sensors would also allow you to determine whether some of the sensors do not in fact contribute meaningfully to overall performance.
- overfeed 1y ago
  > cost & timeline It's really hard to accept cost as the reason when Tesla is preparing a trillion dollar package. I suppose that can be reconciled if the venture is a vehicle (ha!) to shovel money from investors and buyers into Elon's pockets, I imagine the prospect of being the worlds first trillionare is appealing.
- overfeed 1y ago
  > cost & timeline It's really hard to accept cost as the reason when Tesla is preparing a trillion dollar package. I suppose that can be reconciled if one considers the venture to be a vehicle (ha!) to shovel as much money as possible from investors and buyers into Elon's pockets, I imagine the prospect of being the worlds first trillionare is appealing.
- anthem2025 1y ago
  Are people under that impression or are you just repeating the sort of nonsense musk pushes about how sensor fusion is bad?
Ocha 1y ago
Yeap. Same mistake that Boeing did with making redundancy optional upgrade on max8.
- jqpabc123 1y ago
  Another example of what happens when management starts making engineering decisions.
SoftTalker 1y ago
> Vision can fail for a multitude of reasons --- sunlight, rain, darkness, bad road markers, even glare from a dirty windshield. And when it fails, there is no backup plan So like a human driver. Problem is, automatic drivers need to be substantially better than humans to be accepted.
- tarsinge 1y ago
  Humans have a brain though. Current AI is nowhere near that as every engineer know it but common people seem to forget it with all the PR.
brandonagr2 1y ago
Lidar is not a backup to vision, in a waymo both lidar and vision must be working, so you actually have less reliability as now you have two single points of failure.
jmpman 1y ago
Tesla has redundant front facing cameras on their cars. In my 2019 Model 3, there are three front facing cameras, each with varying angles of view, all three behind the rear view mirror, all encased in a small area lined with anti reflective material. Living in an extremely hot climate, that small area, with its anti reflective fuzz have degraded, depositing a film on the window, only in front of the cameras, obscuring all three cameras at the same time. Now, my Tesla just recently started complaining when the sensors were obscured with this deposit, but that wasn’t always the case. I used to be driving down the freeway with autopilot on, and it could barely track. Eventually I looked at the saved video footage and discovered my Tesla was virtually blind, while driving me down the freeway at 85mph. At least now, with recent updates, it warns me that it can’t see very well. However I question the resolution of the sensors. To drive legally in my state, you must have 20/40 vision. When I move my head around, I effectively have 20/40 vision all around my car. If I close 1 eye, I still have 20/40 vision. Does Tesla have effectively 20/40 vision in all 360 degrees? Maybe one of the front facing cameras has optical resolution equal to 20/40, but do the rest of them? I’m skeptical, and expect I’m being driven by what’s equivalent to a human who couldn’t pass the vision test, or at best, a human with just one eye that can pass the vision test. This isn’t even getting into redundancy in the electronics boards, connectivity from the electronics to the CPU, and redundancy in the processsing. We are being asked to put our faith/lives in these non redundant systems, but they’re not designed like Class-A flight critical systems on airplanes.

Someone1234 1y ago

I agree with everything you're saying; but even outside of Tesla, I'd just like to remind people that LIDAR as a complement to vision isn't at all straightforward. Sensor fusion adds real complexity in calibration, time sync, and modeling. Both LIDAR and vision have edge cases where they fail. So you ideally want both, but then the challenge is reconciling disagreements with calibrated, and probabilistic fusion. People seem to be under the mistaken impression that vision is dirty input and LIDAR is somehow clean, when in reality both are noisy inputs with different strengths and weaknesses. I guess my point is: Yes, 100% bring in LIDAR, I believe the future is LIDAR + vision. But when you do that, early iterations can regress significantly from vision-only until the fusion is tuned and calibration is tight, because you have to resolve contradictory data. Ultimately the payoff is higher robustness in exchange for more R&D and development workload (i.e. more cost). The same reason why Tesla needed vision-only to work (cost & timeline) is the same reason why vision+LIDAR is so challenging.

jqpabc123 1y ago
The same reason why Tesla needed vision-only to work (cost & timeline) But vision only hasn't worked --- not as promised, not after a decade's worth of timeline. And it probably won't any time soon either --- for valid engineering reasons. Engineering 101 --- *needing* something to work doesn't make it possible or practical.
ethbr1 1y ago
The primary benefit of multiple sensor fusion from a safety standpoint isn't an absolute decrease in errors. It's the ability to detect sensor disagreements at all. With single modality sensors, you have no way of truly detecting failures in that modality, other than hacks like time-series normalizing (aka expected scenarios). If multiple sensor modalities disagree, even without sensor fusion, you can at least assume something might be awry and drop into a maximum safety operation mode. But we'd think that the budget config of the Boeing 737 MAX would have taught us that tying safety critical systems to single sources of truth is a bad idea... (in that case, critical modality / single physical sensor)
- leoc 1y ago
  > If multiple sensor modalities disagree, even without sensor fusion, you can at least assume something might be awry and drop into a maximum safety operation mode. Also, this is probably when Waymo calls up a human assistant in a developing-country callcentre.
  - ethbr1 1y ago
    Saw that happen a week ago, actually. Non-sensor problem, but a Waymo made a slow right turn too wide, approached the left turning lane of cars, then safed itself by stopping, then remote assistance came online and extricated it.
- AnIrishDuck 1y ago
  > With single modality sensors, you have no way of truly detecting failures in that modality, other than hacks like time-series normalizing (aka expected scenarios). "A man with a watch always knows what time it is. If he gains another, he is never sure" Most safety critical systems actually need at least three redundant sensors. Two is kinda useless: if they disagree, which is right? EDIT: > If multiple sensor modalities disagree, even without sensor fusion, you can at least assume something might be awry and drop into a maximum safety operation mode. This is not always possible. You're on a two lane road. Your vision system tells you there's a pedestrian in your lane. Your LIDAR says the pedestrian is actually in the other lane. There's enough time for a lane change, but not to stop. What do you do?
ra7 1y ago
The complexity argument rings hollow to me. It’s a bit like saying distributed databases are complex because you have to deal with CAP guarantees. Yes, but people still develop them because it has real benefits. It was maybe a valid argument 10 years ago, but in 2025 many companies have shown sensor fusion works just fine. I mean, Waymo has clocked 100M+ miles, so it works. The AV industry has moved on to more interesting problems, while Tesla and Musk are still stuck in the past arguing about sensor choices.
- leoc 1y ago
  Well, it's more like sensor fusion plus extensive human remote intervention, it seems: https://www.nytimes.com/interactive/2024/09/03/technology/zoox-self-driving-cars-remote-control.html https://www.nytimes.com/interactive/2024/09/03/technology/zo... . Mind you, if it takes both LiDAR and call-centre workers to make self-driving work in 2025 and for the foreseeable future, that makes Tesla's old ambition to achieve it with neither look all the more hopeless.
  - ra7 1y ago
    Tesla robotaxis have teleoperation [1], which is worse than remote assistance others use because the operators have direct control. Yet they cannot fully remove safety personnel from the car. The old ambition is dead. [1] https://electrek.co/2025/05/16/tesla-robotaxi-fleet-powered-by-plenty-of-teleoperation/ https://electrek.co/2025/05/16/tesla-robotaxi-fleet-powered-...
  - Narciss 1y ago
    V interesting, thanks for sharing, I didn’t know this
Earw0rm 1y ago
There's no particular reason to use RGB for this kind of machine vision - cognition problem either. Infra-red of a few different wavelengths as well as optical light ranges seems like it'd give a superior result?
- jqpabc123 1y ago
  You've just described some of the rationale for using LIDAR.
maxlin 1y ago
I think you hit the nail on the head - Obviously when Tesla have saturated the potential of vision, they should bring in LiDAR if it can be reasonably added from a hardware point of view. Their current arguments make this clear - it would be surface-level thinking to add LiDAR and the kitchen sink now, complicating the system's evolution and axing scalability. But we're far from plateauing on what can be done with vision - Humans can drive quite well with essentially just sight, so we're far from extinguishing what can be done with it.
baby 1y ago
Sure but if you see something in front of you but LIDAR says "nope I can see 500m away" then you know LIDAR is right
atcon 1y ago
Your comments on sensor fusion seem to describe the weird results of 2 informal ADAS (lidar, vision, lidar + vision, lidar + vision + 4d imaging radar, etc.) “tournaments” conducted earlier this year. There was an earlier HN post about it <https://news.ycombinator.com/item?id=44694891 https://news.ycombinator.com/item?id=44694891> with a comment noting “there was a wide range of crash avoidance behavior even between the same car likely due to the machine learning, and that also makes explaining the differences hard. Hopefully someone with more background on ADAS systems can watch and post what they think.” Notably, sensor confusion is also an “unsolved” problem in humans, eg vision and vestibular (inner ear) conflicts possibly explaining motion sickness/vertigo <https://www.nature.com/articles/s44172-025-00417-2 https://www.nature.com/articles/s44172-025-00417-2> The results of both tournaments: <https://carnewschina.com/2025/07/24/chinas-massive-adas-test-36-cars-15-hazard-scenarios-216-crashes/ https://carnewschina.com/2025/07/24/chinas-massive-adas-test...> Counterintuitively, vision scored best (Tesla Model X) The videos are fascinating to watch (subtitles are available): Tournament 1 (36 cars, 6 Highway Scenarios): <https://www.youtube.com/watch?v=0xumyEf-WRI https://www.youtube.com/watch?v=0xumyEf-WRI> Tournament 2 (26 cars, 9 Urban Scenarios): <https://www.youtube.com/watch?v=GcJnNbm-jUI https://www.youtube.com/watch?v=GcJnNbm-jUI> Highway Scenarios: “tests...included other active vehicles nearby to increase complexity and realism”: <https://electrek.co/2025/07/26/a-chinese-real-world-self-driving-test-36-cars-216-crashes-with-tesla-on-top/ https://electrek.co/2025/07/26/a-chinese-real-world-self-dri...> Urban Scenarios: “a massive, complex roundabout and another segment of road with a few unsignaled intersections and a long straight...The first four tests incorporated portions of this huge roundabout, which would be complex for human drivers, but in situations for which there is quite an obvious solution: don’t hit that car/pedestrian in front of you” <https://electrek.co/2025/07/29/another-huge-chinese-self-driving-test-lawbreaking-indecision-in-urban-driving/ https://electrek.co/2025/07/29/another-huge-chinese-self-dri...>
microtherion 1y ago
> but then the challenge is reconciling disagreements with calibrated, and probabilistic fusion I keep reading arguments like this, but I really don't understand what the problem here is supposed to be. Yes, in a rule based system, this is a challenge, but in an end-to-end neural network, another sensor is just another input, regardless of whether it's another camera, LIDAR, or a sensor measuring the adrenaline level of the driver. If you have enough training data, the model training will converge to a reasonable set of weights for various scenarios. In fact, training data with a richer set of sensors would also allow you to determine whether some of the sensors do not in fact contribute meaningfully to overall performance.
overfeed 1y ago
> cost & timeline It's really hard to accept cost as the reason when Tesla is preparing a trillion dollar package. I suppose that can be reconciled if the venture is a vehicle (ha!) to shovel money from investors and buyers into Elon's pockets, I imagine the prospect of being the worlds first trillionare is appealing.
overfeed 1y ago
> cost & timeline It's really hard to accept cost as the reason when Tesla is preparing a trillion dollar package. I suppose that can be reconciled if one considers the venture to be a vehicle (ha!) to shovel as much money as possible from investors and buyers into Elon's pockets, I imagine the prospect of being the worlds first trillionare is appealing.
anthem2025 1y ago
Are people under that impression or are you just repeating the sort of nonsense musk pushes about how sensor fusion is bad?

Ocha 1y ago

Yeap. Same mistake that Boeing did with making redundancy optional upgrade on max8.

jqpabc123 1y ago
Another example of what happens when management starts making engineering decisions.

SoftTalker 1y ago

> Vision can fail for a multitude of reasons --- sunlight, rain, darkness, bad road markers, even glare from a dirty windshield. And when it fails, there is no backup plan So like a human driver. Problem is, automatic drivers need to be substantially better than humans to be accepted.

tarsinge 1y ago
Humans have a brain though. Current AI is nowhere near that as every engineer know it but common people seem to forget it with all the PR.

brandonagr2 1y ago

Lidar is not a backup to vision, in a waymo both lidar and vision must be working, so you actually have less reliability as now you have two single points of failure.

jmpman 1y ago

Tesla has redundant front facing cameras on their cars. In my 2019 Model 3, there are three front facing cameras, each with varying angles of view, all three behind the rear view mirror, all encased in a small area lined with anti reflective material. Living in an extremely hot climate, that small area, with its anti reflective fuzz have degraded, depositing a film on the window, only in front of the cameras, obscuring all three cameras at the same time. Now, my Tesla just recently started complaining when the sensors were obscured with this deposit, but that wasn’t always the case. I used to be driving down the freeway with autopilot on, and it could barely track. Eventually I looked at the saved video footage and discovered my Tesla was virtually blind, while driving me down the freeway at 85mph. At least now, with recent updates, it warns me that it can’t see very well. However I question the resolution of the sensors. To drive legally in my state, you must have 20/40 vision. When I move my head around, I effectively have 20/40 vision all around my car. If I close 1 eye, I still have 20/40 vision. Does Tesla have effectively 20/40 vision in all 360 degrees? Maybe one of the front facing cameras has optical resolution equal to 20/40, but do the rest of them? I’m skeptical, and expect I’m being driven by what’s equivalent to a human who couldn’t pass the vision test, or at best, a human with just one eye that can pass the vision test. This isn’t even getting into redundancy in the electronics boards, connectivity from the electronics to the CPU, and redundancy in the processsing. We are being asked to put our faith/lives in these non redundant systems, but they’re not designed like Class-A flight critical systems on airplanes.