5 ms·

I don't understand why Musk/Tesla is so insistent on having cars do self-driving purely on cameras. Both from the technical and the marketing point of view hav

I don't understand why Musk/Tesla is so insistent on having cars do self-driving purely on cameras. Both from the technical and the marketing point of view having a car that can do better than any human ever could, even in the most optimal of conditions would be a great thing.

TulliusCicero 2y ago
Musk gets obsessed with big bets when people tell him something isn't possible. That, plus the hardware costs are much saner for Tesla if they don't need a bunch of lidars. That said, this particular big bet looks pretty bad for Musk. He's been confidently promising self-driving either "this year" or "next year" for the last several years, and so far Tesla appears nowhere close to getting L4/L5 driving down.
- gambiting 2y ago
  I mean, Volvo is already delivering the EX90 with a LIDAR integrated from factory for similar money to the Model X - and I can't imagine Volvo is losing money on them, so it can be done. Tesla just chose not to. And yes, I know the EX90 is currently an absolute mess in terms of software and their LIDAR isn't even enabled for driving assists yet, but the hardware is there so it can be built on scale.
  - b112 2y ago
    I presume there is a bigger game here. Androids, eg humanform robots, will require the capacity to navigate on camera alone. Specifically, loads of object detection with smaller objects, imagine an android cleaning off a table after dinner. Visual spectrum, camera identification is vital here. So it's all one ball of wax. Tesla is not juet a car company, likely in 5 years the number of androids they sell will dwarf car sales.
    - zimpenfish 2y ago
      > Androids, eg humanform robots, will require the capacity to navigate on camera alone. But those cameras don't have to be simple visual spectrum cameras, do they? They could have, for example, in each eye, LIDAR, infrared, several focal lengths of visual, etc. Not to mention being able to augment those sensors at places humans just don't get sensor data (e.g: back of head; outward facing from shoulders, hips, etc.; constant presence monitoring via ultrasound / IR / 2.4GHz, etc.)
      - b112 2y ago
        But those cameras don't have to be simple visual spectrum cameras, do they? They do, if they want to do comparative analysis on the trillions upon trillions of existing photographs, and even motion picture frames. Not to mention the entire AI industry is working full out for perfect image recognition, again in our visual spectrum. Something apparent to a person working with openai 5 years ago.
    - blitzar 2y ago
      Ahh 10-d chess.
    - bugglebeetle 2y ago
      Tesla will likely never accomplish anything with robots because they can’t even build a car to typical market standards of quality and robotics is even more greenfield and unrefined in its mass production processes. I guess maybe it’s possible if someone like Shotwell takes over and Musk is relegated to a similar figurehead position.
    - kmarc 2y ago
      Genuine question, why do they require that they navigate on camera alone? I have a robomop which does some pretty accurate scanning with its radar. I thought this is cheap, consumer appliance grade technology already.
    - ywvcbk 2y ago
      > likely in 5 years the number of androids Why would you ever want an unstable bipedal robot? Alternatives seem way more efficient. Also are you predicting that Teslas sales will collapse and they’ll lose most of their EV market share? Because that’s a pretty bold take
  - IshKebab 2y ago
    It looks like the EX90 lidar only points forwards, which is insufficient for full self-driving.
sigmoid10 2y ago
Humans drive with only 2 cameras pointed in a single direction. A Tesla has 8 cameras pointing in every direction. So by default it would already have superhuman awareness. But the difficulty is not perception, it is planning. Turns out to drive among other humans on public roads, you need to correctly understand and predict human behaviour first, which is like half the way to AGI.
- philjohn 2y ago
  Yes - but humans possess other senses, and crucially, our eyes have a much greater dynamic range than even the best cameras (and they are not fitting the absolute best cameras, because that would be even more expensive than LiDAR). It also does away with one of the crucial aspects of "better than human safety" - driving in conditions where a camera is uselss - think moderate to dense fog, heavy rain etc. etc.
  - sigmoid10 2y ago
    We don't have any senses that aren't trivially replicable in a car. Microphones and accelerometers were already plentiful before. And HDR is not really an issue anymore since all carmakers dynamically vary exposure times in their cameras. Other carmakers have tried to put LiDAR and stuff into cars for many years, but it didn't magically give them self-driving either. The key issue to solve is planning - not perception.
    - deleted 2y ago
      [deleted]
    - ywvcbk 2y ago
      > but it didn't magically give them self-driving either. Perhaps. But isn’t Waymo way ahead of Tesla?
      - mewpmewp2 2y ago
        Doesn't Waymo also require manual mapping everything out in detail before it will get to drive there? Street signs etc, etc. So it is Lidar + manual work for each area. While Tesla is trying to go for a solution you can just drop in anywhere.
mschuster91 2y ago
> Both from the technical and the marketing point of view having a car that can do better than any human ever could, even in the most optimal of conditions would be a great thing. Given that the leading causes for road accidents are speeding, DUI and distractions [1], a car doesn't need to be better than any human reasonably can. It just needs to be on the same level as a well-rested, sober human without a phone in their hand. The decision LIDAR vs cameras is the pareto principle in action: what matters is the 80% - the mass that causes the most accidents. [1] https://www.who.int/news-room/fact-sheets/detail/road-traffic-injuries https://www.who.int/news-room/fact-sheets/detail/road-traffi...
a_t48 2y ago
It's cost - LIDARs aren't cheap. Tesla is set up as a consumer car company first and a robotaxi company second, costs matter even more. They could jack up the price of the final product, but presumably it wasn't worth it (at least in his mind?). From a safety perspective more sensors in general are better, if integrated properly. With LIDAR prices are coming down I wonder if he will someday swallow his pride and reintegrate them.
- parl_match 2y ago
  It's almost certainly because he dug the hole so deep in advertising. By saying "we need LIDAR", means that his promise "your car that you own right now will someday be self driving" was a lie. I can't imagine the lawsuits.
  - krick 2y ago
    I cannot imagine how a musk-fan (at this point, I would classify anyone, who believes his claims as a fan) thinks, but it seems so weird to me to actually cling on that hope, that it should make the promise easy to break. It has been a while he made this promise, and it's nowhere near to be delivered. What does it even matter? How long are you comfortable owning the same car (the one that has more electronics in it than iPhone)? 10 years? 20 years? If he doesn't deliver FSD in that time, it doesn't really matter if it will be the same old cars that will achieve FSD. And if you can achieve FSD with better technology on a newer car, it's a win-win, it doesn't really break the promise that your 20 y.o. Tesla will achieve FSD. Maybe in 30 more years it will.
- lucianbr 2y ago
  Why not have at least radar? Plenty of cars have radar for adaptive cruise control and blind spot monitoring, it can't be that expensive. Two sensor kinds would still be an improvement.
  - golol 2y ago
    I think if you have two sensor systems with one being significantly less capable than the other, the gain from having both quickly diminishes. I believe Tesla dropped the radar because it was simply outclassed by the camera in 99% of circumstances. So the radar really just becomes a hacky failsafe for bery rare circumstances, and Musk would like to avoid such a situation, he wants a "clean" solution.
meehai 2y ago
one answer is due to the fact that humans also do this with just 2 pretty bad cameras and a lot of offloading to the cortex. It also simplifies the stack a lot to have a single set of sensors, so the software becomes mostly: getting good training data (iterative loops from failing production cases) and an efficient training algorithm. This scales to more than just AD and also can leverage new breakthroughs from academia
- MrHamburger 2y ago
  With this logic you should also insist on legs instead of wheels.
- Serenacula 2y ago
  I mean, human eyes are in some very important ways better. Not least, they're self cleaning and can adjust their angle for a better view.
  - WA 2y ago
    And they have way better dynamic range.
    - rob74 2y ago
      Yup - if there was a camera today that could consistently (and without any fiddling) reproduce things the way my eyes see them (especially in high dynamic range or low light situations), I would buy it immediately!
- parl_match 2y ago
  Humans have millions of years of evolution in vision processing, and yet we still regularly have 100+ deaths a day in the US, plus many more injuries and fender benders. I don't doubt that it's possible with machine vision alone, but it makes the challenge substantially harder.
- guardian5x 2y ago
  The human eyes are also mounted on a neck that can turn it around in all directions. Kinda useful for parking
velmu 2y ago
Greed. His fortune depends on it.
bnralt 2y ago
I've seen this argument brought up a lot, but I've never seen someone attempt to answer the counterfactual. If LIDAR was added to Tesla's now, how much would FSD improve? I've seen people here who have used it say that with the current software FSD is already pretty good, but my guess is that even with LIDAR they'd still be far from L4. So what kind of difference would we expect to see if, say, tomorrow Tesla announced that they're going to start using LIDAR?
- Eisenstein 2y ago
  Adding an extra sensor onto something not designed for it may not do much. So if that specific question is important to answer, the answer is probably 'not a whole lot', especially since the AI has been training on images only, AFAIK. The problem isn't that the sensors need to be added now, the problem is that without the sensors the cars are half blind and are more likely to make bad decisions and refusing to add them at all shows bad judgement. Rain, snow, mist, fog, darkness -- these are things cameras are very bad at seeing through. Is that thing in front of us solid? Without extensive training on that particular type of object, the AI with the camera has to guess. I'm sure these aren't the best examples -- I am not an expert in LiDAR. But I do have experience with computer vision with camera systems, and they are woefully insufficient for life and death decision making systems when they haven't been trained on the exact specific scenarios they will encounter.
- jimjimjim 2y ago
  My guess is that it's not the lack of sensors that causes the problem with autonomous driving, rather it is the processing of the data. What should the car do if there is a flood covering a dip in the road?
- Fischgericht 2y ago
  The worst thing about my Tesla are the phantom brakes and "emergency lane assist" function (which should be called "steer into opposing traffic for no reason". If I forget to turn those off before starting, my ride will be pure horror. LIDAR probably would help to make Phantom Brakes happen less often, simply because there would be another info source for "is there REALLY an object that is dangerous to me?". Those who say that FSD is "pretty good" are living in a fantasy world. There is hard data on miles between critical disengagements (which really should be called "if the driver doesn't respond within a fraction of a second, people will die"), and depending on region, model, weather etc it's between 13 and 115 miles right now. Over here in Germany there are statistics that a human driver will have the equivalent every 155,000 miles. "Pretty good" just doesn't cut it when it's about the risk of killing people. My Model 3 right now detects about 60-70% of school children crossing the road (keep in mind roads in Germany are narrow, and humans including kids are using the roads, too). 30%-40% of those I would kill every morning on my way to the office. And the thing is: 70% isn't enough for this, 80% isn't, 90% isn't, 99% isn't, and 99,999% isn't. Side note: People constantly claim that Waymo is autonomous. It's as autonomous as a tram. They only work because it the cities they operate every single road they use have been mapped by hand and is constantly updated. Send a Waymo into my city over here, and will also kill a couple of kids per day. Years ahead of Tesla? Yes. Good enough? Hell no.
m463 2y ago
The history of technology is pretty much directly opposed to your wishes. The prosperity of technology basically solves the problem with the most terrible hardware possible. For example, if you have a really good microprocessor that works, what you should do is shrink it until it barely works or speed it up until it barely works to get a similar version for cheaper or faster version for the same price. Same goes with cars. If you have a car that works with ultrasonic and lidar and cameras, eliminate the expensive sensors and reduce the cost. Reducing the cost will let more people afford the cars, and sell more cars. To Musk that is basically his plan all along - to replace ICE cars with EVs. Just wish we could keep turn signal/wiper/headlight stalks.
- jahnu 2y ago
  > If you have a car that works with ultrasonic and lidar and cameras Well... that conditional clause was never satisfied in the first place ;)
  - chpatrick 2y ago
    Waymo works pretty damn well.
    - jahnu 2y ago
      Sure! I would love to try that too. I was referring to Tesla. They never had a fully or nearly fully working system that they could iterate on.
    - simiones 2y ago
      It does in certain strictly determined conditions + human assistance. But the point Tesla didn't go through "FSD with ultrasonic + lidar + cameras", and then start to reduce the sensors. They went from "no FSD" to "broken non-working FSD, but now only with cameras". That is not how you strip down a working product to reduce costs.
      - vardump 2y ago
stonewhite 2y ago
This is a good practice to reduce at least some amount of dependency to outside US. Also this is the definition of innovation; making high tech out of low tech.
elorant 2y ago
Because he needs to maintain market value. Musk is part inventor and part salesman. He needs high valuations to keep risky endeavors afloat. What's better than sell the future.
bufferoverflow 2y ago
Computers should be able to drive better than humans with just vision, simply because of * much faster reaction times * ability to model multiple possible trajectories * ability to see in all directions simultaneously * no distractions, no tiredness, no falling asleep, no health problems

TulliusCicero 2y ago

Musk gets obsessed with big bets when people tell him something isn't possible. That, plus the hardware costs are much saner for Tesla if they don't need a bunch of lidars. That said, this particular big bet looks pretty bad for Musk. He's been confidently promising self-driving either "this year" or "next year" for the last several years, and so far Tesla appears nowhere close to getting L4/L5 driving down.

gambiting 2y ago
I mean, Volvo is already delivering the EX90 with a LIDAR integrated from factory for similar money to the Model X - and I can't imagine Volvo is losing money on them, so it can be done. Tesla just chose not to. And yes, I know the EX90 is currently an absolute mess in terms of software and their LIDAR isn't even enabled for driving assists yet, but the hardware is there so it can be built on scale.
- b112 2y ago
  I presume there is a bigger game here. Androids, eg humanform robots, will require the capacity to navigate on camera alone. Specifically, loads of object detection with smaller objects, imagine an android cleaning off a table after dinner. Visual spectrum, camera identification is vital here. So it's all one ball of wax. Tesla is not juet a car company, likely in 5 years the number of androids they sell will dwarf car sales.
  - zimpenfish 2y ago
    > Androids, eg humanform robots, will require the capacity to navigate on camera alone. But those cameras don't have to be simple visual spectrum cameras, do they? They could have, for example, in each eye, LIDAR, infrared, several focal lengths of visual, etc. Not to mention being able to augment those sensors at places humans just don't get sensor data (e.g: back of head; outward facing from shoulders, hips, etc.; constant presence monitoring via ultrasound / IR / 2.4GHz, etc.)
    - b112 2y ago
      But those cameras don't have to be simple visual spectrum cameras, do they? They do, if they want to do comparative analysis on the trillions upon trillions of existing photographs, and even motion picture frames. Not to mention the entire AI industry is working full out for perfect image recognition, again in our visual spectrum. Something apparent to a person working with openai 5 years ago.
  - blitzar 2y ago
    Ahh 10-d chess.
  - bugglebeetle 2y ago
    Tesla will likely never accomplish anything with robots because they can’t even build a car to typical market standards of quality and robotics is even more greenfield and unrefined in its mass production processes. I guess maybe it’s possible if someone like Shotwell takes over and Musk is relegated to a similar figurehead position.
  - kmarc 2y ago
    Genuine question, why do they require that they navigate on camera alone? I have a robomop which does some pretty accurate scanning with its radar. I thought this is cheap, consumer appliance grade technology already.
  - ywvcbk 2y ago
    > likely in 5 years the number of androids Why would you ever want an unstable bipedal robot? Alternatives seem way more efficient. Also are you predicting that Teslas sales will collapse and they’ll lose most of their EV market share? Because that’s a pretty bold take
- IshKebab 2y ago
  It looks like the EX90 lidar only points forwards, which is insufficient for full self-driving.

sigmoid10 2y ago

Humans drive with only 2 cameras pointed in a single direction. A Tesla has 8 cameras pointing in every direction. So by default it would already have superhuman awareness. But the difficulty is not perception, it is planning. Turns out to drive among other humans on public roads, you need to correctly understand and predict human behaviour first, which is like half the way to AGI.

philjohn 2y ago
Yes - but humans possess other senses, and crucially, our eyes have a much greater dynamic range than even the best cameras (and they are not fitting the absolute best cameras, because that would be even more expensive than LiDAR). It also does away with one of the crucial aspects of "better than human safety" - driving in conditions where a camera is uselss - think moderate to dense fog, heavy rain etc. etc.
- sigmoid10 2y ago
  We don't have any senses that aren't trivially replicable in a car. Microphones and accelerometers were already plentiful before. And HDR is not really an issue anymore since all carmakers dynamically vary exposure times in their cameras. Other carmakers have tried to put LiDAR and stuff into cars for many years, but it didn't magically give them self-driving either. The key issue to solve is planning - not perception.
  - deleted 2y ago
    [deleted]
  - ywvcbk 2y ago
    > but it didn't magically give them self-driving either. Perhaps. But isn’t Waymo way ahead of Tesla?
    - mewpmewp2 2y ago
      Doesn't Waymo also require manual mapping everything out in detail before it will get to drive there? Street signs etc, etc. So it is Lidar + manual work for each area. While Tesla is trying to go for a solution you can just drop in anywhere.

mschuster91 2y ago

> Both from the technical and the marketing point of view having a car that can do better than any human ever could, even in the most optimal of conditions would be a great thing. Given that the leading causes for road accidents are speeding, DUI and distractions [1], a car doesn't need to be better than any human reasonably can. It just needs to be on the same level as a well-rested, sober human without a phone in their hand. The decision LIDAR vs cameras is the pareto principle in action: what matters is the 80% - the mass that causes the most accidents. [1] https://www.who.int/news-room/fact-sheets/detail/road-traffic-injuries https://www.who.int/news-room/fact-sheets/detail/road-traffi...

a_t48 2y ago

It's cost - LIDARs aren't cheap. Tesla is set up as a consumer car company first and a robotaxi company second, costs matter even more. They could jack up the price of the final product, but presumably it wasn't worth it (at least in his mind?). From a safety perspective more sensors in general are better, if integrated properly. With LIDAR prices are coming down I wonder if he will someday swallow his pride and reintegrate them.

parl_match 2y ago
It's almost certainly because he dug the hole so deep in advertising. By saying "we need LIDAR", means that his promise "your car that you own right now will someday be self driving" was a lie. I can't imagine the lawsuits.
- krick 2y ago
  I cannot imagine how a musk-fan (at this point, I would classify anyone, who believes his claims as a fan) thinks, but it seems so weird to me to actually cling on that hope, that it should make the promise easy to break. It has been a while he made this promise, and it's nowhere near to be delivered. What does it even matter? How long are you comfortable owning the same car (the one that has more electronics in it than iPhone)? 10 years? 20 years? If he doesn't deliver FSD in that time, it doesn't really matter if it will be the same old cars that will achieve FSD. And if you can achieve FSD with better technology on a newer car, it's a win-win, it doesn't really break the promise that your 20 y.o. Tesla will achieve FSD. Maybe in 30 more years it will.
lucianbr 2y ago
Why not have at least radar? Plenty of cars have radar for adaptive cruise control and blind spot monitoring, it can't be that expensive. Two sensor kinds would still be an improvement.
- golol 2y ago
  I think if you have two sensor systems with one being significantly less capable than the other, the gain from having both quickly diminishes. I believe Tesla dropped the radar because it was simply outclassed by the camera in 99% of circumstances. So the radar really just becomes a hacky failsafe for bery rare circumstances, and Musk would like to avoid such a situation, he wants a "clean" solution.

meehai 2y ago

one answer is due to the fact that humans also do this with just 2 pretty bad cameras and a lot of offloading to the cortex. It also simplifies the stack a lot to have a single set of sensors, so the software becomes mostly: getting good training data (iterative loops from failing production cases) and an efficient training algorithm. This scales to more than just AD and also can leverage new breakthroughs from academia

MrHamburger 2y ago
With this logic you should also insist on legs instead of wheels.
Serenacula 2y ago
I mean, human eyes are in some very important ways better. Not least, they're self cleaning and can adjust their angle for a better view.
- WA 2y ago
  And they have way better dynamic range.
  - rob74 2y ago
    Yup - if there was a camera today that could consistently (and without any fiddling) reproduce things the way my eyes see them (especially in high dynamic range or low light situations), I would buy it immediately!
parl_match 2y ago
Humans have millions of years of evolution in vision processing, and yet we still regularly have 100+ deaths a day in the US, plus many more injuries and fender benders. I don't doubt that it's possible with machine vision alone, but it makes the challenge substantially harder.
guardian5x 2y ago
The human eyes are also mounted on a neck that can turn it around in all directions. Kinda useful for parking

velmu 2y ago

Greed. His fortune depends on it.

bnralt 2y ago

I've seen this argument brought up a lot, but I've never seen someone attempt to answer the counterfactual. If LIDAR was added to Tesla's now, how much would FSD improve? I've seen people here who have used it say that with the current software FSD is already pretty good, but my guess is that even with LIDAR they'd still be far from L4. So what kind of difference would we expect to see if, say, tomorrow Tesla announced that they're going to start using LIDAR?

Eisenstein 2y ago
Adding an extra sensor onto something not designed for it may not do much. So if that specific question is important to answer, the answer is probably 'not a whole lot', especially since the AI has been training on images only, AFAIK. The problem isn't that the sensors need to be added now, the problem is that without the sensors the cars are half blind and are more likely to make bad decisions and refusing to add them at all shows bad judgement. Rain, snow, mist, fog, darkness -- these are things cameras are very bad at seeing through. Is that thing in front of us solid? Without extensive training on that particular type of object, the AI with the camera has to guess. I'm sure these aren't the best examples -- I am not an expert in LiDAR. But I do have experience with computer vision with camera systems, and they are woefully insufficient for life and death decision making systems when they haven't been trained on the exact specific scenarios they will encounter.
jimjimjim 2y ago
My guess is that it's not the lack of sensors that causes the problem with autonomous driving, rather it is the processing of the data. What should the car do if there is a flood covering a dip in the road?
Fischgericht 2y ago
The worst thing about my Tesla are the phantom brakes and "emergency lane assist" function (which should be called "steer into opposing traffic for no reason". If I forget to turn those off before starting, my ride will be pure horror. LIDAR probably would help to make Phantom Brakes happen less often, simply because there would be another info source for "is there REALLY an object that is dangerous to me?". Those who say that FSD is "pretty good" are living in a fantasy world. There is hard data on miles between critical disengagements (which really should be called "if the driver doesn't respond within a fraction of a second, people will die"), and depending on region, model, weather etc it's between 13 and 115 miles right now. Over here in Germany there are statistics that a human driver will have the equivalent every 155,000 miles. "Pretty good" just doesn't cut it when it's about the risk of killing people. My Model 3 right now detects about 60-70% of school children crossing the road (keep in mind roads in Germany are narrow, and humans including kids are using the roads, too). 30%-40% of those I would kill every morning on my way to the office. And the thing is: 70% isn't enough for this, 80% isn't, 90% isn't, 99% isn't, and 99,999% isn't. Side note: People constantly claim that Waymo is autonomous. It's as autonomous as a tram. They only work because it the cities they operate every single road they use have been mapped by hand and is constantly updated. Send a Waymo into my city over here, and will also kill a couple of kids per day. Years ahead of Tesla? Yes. Good enough? Hell no.

m463 2y ago

The history of technology is pretty much directly opposed to your wishes. The prosperity of technology basically solves the problem with the most terrible hardware possible. For example, if you have a really good microprocessor that works, what you should do is shrink it until it barely works or speed it up until it barely works to get a similar version for cheaper or faster version for the same price. Same goes with cars. If you have a car that works with ultrasonic and lidar and cameras, eliminate the expensive sensors and reduce the cost. Reducing the cost will let more people afford the cars, and sell more cars. To Musk that is basically his plan all along - to replace ICE cars with EVs. Just wish we could keep turn signal/wiper/headlight stalks.

jahnu 2y ago
> If you have a car that works with ultrasonic and lidar and cameras Well... that conditional clause was never satisfied in the first place ;)
- chpatrick 2y ago
  Waymo works pretty damn well.
  - jahnu 2y ago
    Sure! I would love to try that too. I was referring to Tesla. They never had a fully or nearly fully working system that they could iterate on.
  - simiones 2y ago
    It does in certain strictly determined conditions + human assistance. But the point Tesla didn't go through "FSD with ultrasonic + lidar + cameras", and then start to reduce the sensors. They went from "no FSD" to "broken non-working FSD, but now only with cameras". That is not how you strip down a working product to reduce costs.
    - vardump 2y ago

stonewhite 2y ago

This is a good practice to reduce at least some amount of dependency to outside US. Also this is the definition of innovation; making high tech out of low tech.

elorant 2y ago

Because he needs to maintain market value. Musk is part inventor and part salesman. He needs high valuations to keep risky endeavors afloat. What's better than sell the future.

bufferoverflow 2y ago

Computers should be able to drive better than humans with just vision, simply because of * much faster reaction times * ability to model multiple possible trajectories * ability to see in all directions simultaneously * no distractions, no tiredness, no falling asleep, no health problems