11 ms·

Tesla's decision not to use the LIDAR as a safety feature (i.e. having reliable high-resolution data about things the car can collide with) is so incredibly ind

Tesla's decision not to use the LIDAR as a safety feature (i.e. having reliable high-resolution data about things the car can collide with) is so incredibly indefensible, since solving the last 1% of this using only vision likely requires a general artificial intelligence Prediction: Tesla will be the last of all major auto manufacturers to get to L5 autonomy. Time interval between when Tesla L5 FSD is finally available and when humanity is destroyed by the general AI it runs on will be very awesome and also very short

soheil 5y ago
> is so incredibly indefensible How can you make such a strong statement when you simply don't know how to achieve full autonomy? This reminds of the teapot orbiting the Sun argument [0]. You and people defending Lidar by their teeth don't sound too different from religious zealots who've "seen the light". [0] https://en.wikipedia.org/wiki/Russell%27s_teapot https://en.wikipedia.org/wiki/Russell%27s_teapot
SheinhardtWigCo 5y ago
They’re betting that they can use a massive feedback loop to train a set of neural networks to the point where they are as accurate as LiDAR without actually firing any lasers. Even if you believe this goal is possible to achieve at some point in the future, I think the argument falls apart when you consider that it will take years, probably decades, for a pure vision approach to catch up to where Waymo is today in terms of safety. (They have cameras too.) That Tesla can’t afford to fit expensive LiDAR sensors to all of the cars it sells is Tesla’s problem. Regulators won’t give a shit that pure vision is “better” in theory. They will simply compare Tesla’s crash rate in autonomous mode with that of Waymo and other AV operators, and act accordingly.
- DSingularity 5y ago
  Definitely. Especially with car companies like NIO strapping in LIDAR to their upcoming models.
- bluepanda928752 5y ago
  I understand why they made the "no-LIDAR" bet early when the LIDARs were completely unpractical for a production consumer car However, nowadays it starts to look that 100% reliable depth estimation from cameras might actually require a human-level AI to work and also solid-state LIDAR technology is becoming cheap enough and integrateable into normal cars, but Tesla can't really change their stance on this without admitting that FSD options they already sold would not actually become FSD within the lifetimes of these vehicles. I suspect this might also be the reason why Karpathy looks more and more nervous with each new talk
  - anitil 5y ago
    Is it something that can be retrofitted on existing models?
    - bluepanda928752 5y ago
      LIDARs unlikely since they require a line of sight forwards, backwards and possibly on the sides of the vehicle
  - donio 5y ago
    > FSD options they already sold would not actually become FSD within the lifetimes of these vehicles That's pretty much a given at this point but they will not admit it until a class-action lawsuit forces them to.
    - bluepanda928752 5y ago
      But we might actually get a functioning Mars base out of this since that's where Musk will be hiding when the FTC finally wakes up /s
  - trhway 5y ago
    >100% reliable depth estimation from cameras might actually require a human-level AI to work you don't need 100%, and even humans are far from 100% (500Mpx resolution of our eyes allows to basically sheer brute force through it in many cases). The stereo setup provides great and fast estimation with several megapixel resolution with good fps (way better than lidar) for majority of situations. It is some share of the [part of the] scenes, and you really know it right then and there, where you need AI and/or very sophisticated compute heavy algorithms. So instead of throwing AI and the compute power at those parts, you just pull the points from the lidar (and even radar if the things are that bad) covering that segment. And that way, given a couple more iterations of sensors (from current 20Mpx+ to the hundreds Mpx) and compute, it will be doing even better than humans. Anybody not doing sensor fusion would be a loser though - just like going into a fist fight with one hand intentionally disabled.
- practice9 5y ago
  > I think the argument falls apart when you consider that it will take years, probably decades, for a pure vision approach to catch up to where Waymo is today in terms of safety. (They have cameras too.) On what set of metrics do you think Waymo is safer? IMO it's too early to compare and cherry-picked proofs both from Waymo and Tesla are not really representative.
  - ra7 5y ago
    Waymo has published detailed safety performance data of their Arizona operations: https://waymo.com/safety/performance-data https://waymo.com/safety/performance-data You can read their other safety whitepapers in https://waymo.com/safety https://waymo.com/safety
    - gibolt 5y ago
      Arizona roads are also mapped to extreme precision, have very wide lanes, and are optimized for cars. Waymo has prioritized low intervention by being overly cautious and avoiding hard maneuvers (like many left turns). That doesn't work when they scale up to any other set of normal roads, especially as density and complexity increases.
      - ra7 5y ago
        They don't avoid left turns. There are plenty of videos from Chandler, AZ of Waymo performing unprotected left turns perfectly fine. They will always map roads to precision, whether it's Arizona or San Francisco. Why is that a problem? You should either look at their CA disengagement reports over the years or wait until they roll out a service in SF (where they've been testing heavily). That will show how safe they are in dense environments.
        olau 5y ago
        From what I gather, they manually mark sections as hard when the cars get stuck there, e.g. due to road work, and then their routing system chooses another route, e.g. one that avoids the left turn. The video with the Waymo car getting stuck and taking off from the rescue team had an example of this. I guess it makes perfect sense from a engineering perspective.
- jfim 5y ago
  One of the gains from using lidar is also that it's a different sensor altogether from cameras, with different failure modes. For example, cameras are sensitive to glare from reflections (sun near sunset or reflecting on metallic objects) and oncoming traffic at night. Lidars operate on a different narrower wavelength and are unlikely to be affected by that, although they might struggle with objects that have low reflectivity at a long distance. The fact that these sensors are different means that the intersection where a dangerous situation would not be detected by either sensor is much smaller than any sensor individually. In any case, once AVs are deployed at scale, if it becomes apparent that some sensors can be removed or replaced by something else, then they will be if there's a case for it.
ra7 5y ago
> Prediction: Tesla will be the last of all major auto manufacturers to get to L5 autonomy. Tesla is also the only company to claim to target L5 autonomy. Everyone else, including Waymo, is strictly targeting L4 and say L5 autonomy is not possible or realistic. L5 is a pipe dream.
liuliu 5y ago
You get sparse point cloud from LIDAR sensors, not accurate 3D maps. This is the main reason why some people think LIDAR may not work well (mostly, only comma.ai and Tesla folks). Vision can also get you 3D maps, either in active manner (IR floodlight or structured lighting) or not. I will reserve my judgement until see more from either side.
- bluepanda928752 5y ago
  LIDAR is valuable as a safety feature, i.e. it can (unlike radars or cameras) reliably see (at least in clear weather) if there's anything in the car's path warranting evasion/braking maneuvers. In particular it's important that the LIDAR is dumb, i.e. its failure mores are predictable
  - liuliu 5y ago
    LIDAR operates by timing how a photon reflects from a surface, it doesn't guarantee see everything. As you stated already, it cannot see in snow or rain. I am actually on sensor-fusion side, but I don't think we should jump to the conclusion LIDAR is the best 3D mapping method. For one example, a truck has a breaking distance of 600ft, either Velodyne or OS1 LIDAR has range less than that.
    - bluepanda928752 5y ago
      I agree about LIDARs not being the best general 3D mapping method, my point was mostly about using it as a dumb physics-based safety system. For autonomous trucks, limited LIDAR range could be mitigated by reducing speed accordingly and/or employing more powerful (or SWIR) LIDARs since trucks are more expensive and there could be a bigger budget for sensors
- ra7 5y ago
  > You get sparse point cloud from LIDAR sensors Take a look at how dense the point cloud from Waymo's 5th gen LIDAR is: https://www.youtube.com/watch?v=COgEQuqTAug&t=11599s https://www.youtube.com/watch?v=COgEQuqTAug&t=11599s. They just talked about this a few days ago.
etrautmann 5y ago
The vision-only strategy only made a shred of sense before high performance / low cost LIDARs like the Ouster OS1 became available. Now it’s an indefensible position on safety or economic grounds.
- pokerhobo 5y ago
  LIDAR in a drizzly city like Seattle is of minimal value so the system has to fallback to vision only anyways
  - rpmisms 5y ago
    It's an anti-fragile approach, a term you'll recognize if you know of Nassim Taleb and his work. I think it will win in the long run, because not requiring HD Maps or specialized sensors is an advantage, and even if it takes more resources to make it work initially, it will save billions in the future, assuming, of course, it ever ships. Obviously, pure vision is a viable system, as it's what we as humans use. The question remains as to whether or not it will be comparable to more precise LiDAR based systems in the near future.
    - etrautmann 5y ago
      This is not a good argument overall for several reasons. First, we should aim to greatly exceed human performance on safety. Second, whatever work goes into making the algorithms work well using vision can just as easily still inform a system that fuses that data with LIDAR for enhanced situational awareness and safety.
tokipin 5y ago
I predict the opposite. Tesla sold half a million cars last year and will sell nearly one million this year. The data they have access to is increasing by orders of magnitude. I bet there is a point, let's say 20 million cars total, where they can pull so much high quality data that they will be able to surpass lidar capabilities for the purposes of self driving. The lidar/no lidar discussion is a fun one because people have different ideas about how the world works. Personally I think LiDAR is the modern version of expert systems. It appeals to a logical/geometric intuition but the approach is brittle to real world contact, especially when paired with HD maps which are a great way to drive yourself into a local maximum.
- 01100011 5y ago
  The fallacy here is that the scale of the neural network used by Tesla is sufficient to capture the problem of driving given enough training. There is no guarantee that a reasonably priced neural network can encompass the task of driving. Having training data beyond a certain point is overrated, and Tesla's advantage in gathering it is overstated. Other companies are capturing this data as well. Is there any indication that the data Tesla is collecting is of a higher value, or is it just more bytes?
  - daveguy 5y ago
    I'd say the data that Tesla collects is of lower value, because it doesn't have sensor info from a different modality. Other companies are getting a good reference to ground truth for both camera to lidar and lidar to camera. I don't know how much more valuable accurate distance sensing over a 3d field is compared to not having it, but I so know it's more valuable. It may be valuable enough to require a few petaflops less computing power.
  - aimkey 5y ago
    It seems as if the people gobbling up the "Tesla has the data! Autopilot will keep getting better!" line have never trained a neural network in their life. Models converge. Loss stops decreasing, regardless of more incoming data. Extreme manual data cleaning effort becomes required to prevent overfitting. Model architecture has to change and hyper parameters have to be tweaked. Then you're back at square one as far as testing goes if you change any of those things. The notion that Tesla's model HAS to keep improving simply because they will be able to pile on more (unlabeled!) data is laughably false. And, in fact, quite insulting to the intelligence of even the most casual ML engineers.
codeulike 5y ago
Innovation is a gamble. You're not wrong to point out they might fail. The likeliness of failure is what makes it worth the gamble of trying.
- bobsomers 5y ago
  Generally this is true, but in the safety critical domain you don't gamble. You do your homework and make sure you aren't exposing your users to unnecessary levels of risk. If Tesla was developing their system with trained safety drivers or on closed courses, I think they would have higher moral ground to gamble here. But placing the untrained public behind the wheel of alpha quality software is unethical IMHO. There ways to develop autonomous software that are significantly less risky, and the only reason Tesla is doing it this way is for marketing/PR purposes as far as I can tell.
dheera 5y ago
My prediction is that Tesla will eventually use LIDAR despite whatever they are saying now. Right now their profit model is selling cars, not autonomy, so everything is optimized for that, including the decision to not use LIDAR.
- sadfasf122 5y ago
  lol what? did you watch the video at all
  - dheera 5y ago
    Well yes, but I think they'll be singing a different tune in 5 years, and even more so when more Tesla cars actually start driving in weather conditions less favorable than Palo Alto and Austin. At some point when LIDAR is cheap enough there is no reason for Elon Musk to not give in and use them. Right now he's constraining the problem to the cost of the car.
    - minhazm 5y ago
      LiDAR is not better in worse weather conditions. LiDAR performance degrades in rain and snow. That’s where something like radar is better. The Tesla get is that humans drive with vision so they should be able to as well. Also every other self driving solution must solve the vision problem also in order to be successful. LiDAR doesn’t tell you that it’s a bag on the road vs a raccoon. So the question is that once you solve the vision problem, do you still need LiDAR for any meaningful impact?
      - dheera 5y ago
        > LiDAR doesn’t tell you that it’s a bag on the road vs a raccoon I think that's largely an issue with the early LIDAR devices today, but not necessarily what may be to come. There's something to be said about measuring actual data with solid physics vs. inferring distances with billions of operations on RGB data. If you were landing a commercial aircraft in fog, you most certainly don't rely on your eyes to do most of it, but it is in fact possible to do safely precisely because we do have good sensors on them. I fully agree with leveraging the scale and maturity of RGB sensors for cars today, the talk is spot on about that, but that's (a) circling back to the fact that Tesla needs to sell cars now not next year and (b) not a good case against use of LIDAR in the future.
cs702 5y ago
I strongly disagree. By all measures I've seen (including a couple of slides in the OP's video), Tesla's self-driving is far safer than human driving: the number of accidents and deaths per mile driven are something like an order of magnitude lower (i.e., around 10x safer). I mean, the machine never gets distracted, tired, sleepy, emotional, drunk, etc., so it is a LOT LESS likely to crash on boring, monotonous road segments than most people -- who do get distracted, tired, sleepy, etc. Not only that, but people make really scary mistakes in routine circumstances. The video shows several examples of human drivers hitting the accelerator when they actually meant to hit the brake! The criticism of autopilot is really about it getting tripped-up in response to statistically rare, unusual circumstances, i.e., edge cases. Karpathy et al are working on getting better at those, bringing the rate of situations that surprise autopilot closer and closer to 0%, even if it can never be achieved -- there will be always be surprises. Personally, I would rather take a tiny risk of crash on rare, once-in-a-million-miles events with autopilot driving than a ~1% risk of crash per 1000 to 2000 miles with everyday human driving. Prediction: Tesla will be the first of all major automakers to get to level 4 and 5 autonomy.
- AndrewKemendo 5y ago
  >Tesla's self-driving is far safer than human driving: the number of accidents and deaths per mile driven are something like an order of magnitude lower Still not close to good enough for people to accept: "Participants from both countries required Self Driving Vehicles to be 4-5 times as safe as Human Driven Vehicles" [0] 0: https://pubmed.ncbi.nlm.nih.gov/32202821/ https://pubmed.ncbi.nlm.nih.gov/32202821/
  - deleted 5y ago
    [deleted]
  - minhazm 5y ago
    It’s already about 10x. But it is a bit tricky since they don’t break the numbers out by highway and local driving. Also their active safety features outside of AP also improve safety. So does it need to be 4-5x better than an already improved system that has AEB and lane departure avoidance and other safety features? https://www.tesla.com/VehicleSafetyReport https://www.tesla.com/VehicleSafetyReport > In the 1st quarter, we registered one accident for every 4.19 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot but with our active safety features, we registered one accident for every 2.05 million miles driven. For those driving without Autopilot and without our active safety features, we registered one accident for every 978 thousand miles driven. By comparison, NHTSA’s most recent data shows that in the United States there is an automobile crash every 484,000 miles.
paxys 5y ago
> Prediction: Tesla will be the last of all major auto manufacturers to get to L5 autonomy That doesn't mean much considering no company is getting to L5 autonomy likely in our lifetime and possibly beyond.
vgchh 5y ago
Ultimately the proof is in the pudding. With Tesla FSD you can drive the highways from New York to Boston without any issues. I am sure there are many more routes across the country that can also be driven like that. Definitely not L5, but it works. Yet to see that from any other automaker, LIDAR or not. As far as I am concerned, great job Tesla! Keep it up, I am sure you will work through more tougher problems.
kemiller 5y ago
Waymo etc do not use LIDAR for object sensing, only for positioning. LIDAR sucks for object sensing because it gives you no information about whether it's a plastic bag or a person — you still need vision for that. Even if you just err on the safe side and brake, that itself can cause an accident unecessarily.
- jowday 5y ago
  You're wrong on that. LIDAR is used for object detection in every self driving car company that uses LIDAR. There's tons of research on it. Previous generations of LIDAR were not great at classifying small objects and road debris but it works great for detecting cars, pedestrians, etc. https://paperswithcode.com/task/3d-object-detection https://paperswithcode.com/task/3d-object-detection Next-gen LIDAR has great density, and I bet it would do a decent job differentiating between a plastic bag or a rock in the middle of the road. In addition to depth LIDAR also returns intensity and several other metrics, which can be used as input to an ML model. It's why you can read the lettering on the side of the semi truck in this video of Waymo's next-gen LIDAR. https://youtu.be/COgEQuqTAug?t=11601 https://youtu.be/COgEQuqTAug?t=11601
ec109685 5y ago
Why do you think the last 1% is dependent on LIDAR versus any of the other multitude of gaps between today’s autonomy and L5? If the only way it becomes practical to achieve L5 is to use LIDAR, Tesla can obviously add it. But if they waited until LIDAR was cheap and practical, they still wouldn’t be shipping any hardware doing autonomy today, and not collecting the data needed to train their models and delivering value today. Also, with vision based systems, it operates in somewhat an intuitive fashion given we have eyes too.
likearocket 5y ago
No one is going to get to a true L5 for a long time. That is totally irrelevant. It's a war of attrition. Whoever can monetize L3/L4 and can scale without any vehicle upgrade cost is going to win. It's pretty obvious lidar is very very silly since it doesn't scale. It is also pretty easy to see that Tesla doesn't have to hit L5 to have won autonomy. It just has to successfully monetize L3/L4.
jsight 5y ago
> solving the last 1% of this using only vision likely requires a general artificial intelligence That is likely close to true with lidar as well. See also some of Waymo's recent struggles in unexpected construction zones. Maybe lidar helps in getting there, but I'm afraid they all hit a pretty tough ceiling without this.
- ra7 5y ago
  > That is likely close to true with lidar as well. See also some of Waymo's recent struggles in unexpected construction zones. That example has nothing to do with lidar, but with planning. The car was able to detect the construction zone just fine.
  - jsight 5y ago
    That's what I said.

soheil 5y ago

> is so incredibly indefensible How can you make such a strong statement when you simply don't know how to achieve full autonomy? This reminds of the teapot orbiting the Sun argument [0]. You and people defending Lidar by their teeth don't sound too different from religious zealots who've "seen the light". [0] https://en.wikipedia.org/wiki/Russell%27s_teapot https://en.wikipedia.org/wiki/Russell%27s_teapot

SheinhardtWigCo 5y ago

They’re betting that they can use a massive feedback loop to train a set of neural networks to the point where they are as accurate as LiDAR without actually firing any lasers. Even if you believe this goal is possible to achieve at some point in the future, I think the argument falls apart when you consider that it will take years, probably decades, for a pure vision approach to catch up to where Waymo is today in terms of safety. (They have cameras too.) That Tesla can’t afford to fit expensive LiDAR sensors to all of the cars it sells is Tesla’s problem. Regulators won’t give a shit that pure vision is “better” in theory. They will simply compare Tesla’s crash rate in autonomous mode with that of Waymo and other AV operators, and act accordingly.

DSingularity 5y ago
Definitely. Especially with car companies like NIO strapping in LIDAR to their upcoming models.
bluepanda928752 5y ago
I understand why they made the "no-LIDAR" bet early when the LIDARs were completely unpractical for a production consumer car However, nowadays it starts to look that 100% reliable depth estimation from cameras might actually require a human-level AI to work and also solid-state LIDAR technology is becoming cheap enough and integrateable into normal cars, but Tesla can't really change their stance on this without admitting that FSD options they already sold would not actually become FSD within the lifetimes of these vehicles. I suspect this might also be the reason why Karpathy looks more and more nervous with each new talk
- anitil 5y ago
  Is it something that can be retrofitted on existing models?
  - bluepanda928752 5y ago
    LIDARs unlikely since they require a line of sight forwards, backwards and possibly on the sides of the vehicle
- donio 5y ago
  > FSD options they already sold would not actually become FSD within the lifetimes of these vehicles That's pretty much a given at this point but they will not admit it until a class-action lawsuit forces them to.
  - bluepanda928752 5y ago
    But we might actually get a functioning Mars base out of this since that's where Musk will be hiding when the FTC finally wakes up /s
- trhway 5y ago
  >100% reliable depth estimation from cameras might actually require a human-level AI to work you don't need 100%, and even humans are far from 100% (500Mpx resolution of our eyes allows to basically sheer brute force through it in many cases). The stereo setup provides great and fast estimation with several megapixel resolution with good fps (way better than lidar) for majority of situations. It is some share of the [part of the] scenes, and you really know it right then and there, where you need AI and/or very sophisticated compute heavy algorithms. So instead of throwing AI and the compute power at those parts, you just pull the points from the lidar (and even radar if the things are that bad) covering that segment. And that way, given a couple more iterations of sensors (from current 20Mpx+ to the hundreds Mpx) and compute, it will be doing even better than humans. Anybody not doing sensor fusion would be a loser though - just like going into a fist fight with one hand intentionally disabled.
practice9 5y ago
> I think the argument falls apart when you consider that it will take years, probably decades, for a pure vision approach to catch up to where Waymo is today in terms of safety. (They have cameras too.) On what set of metrics do you think Waymo is safer? IMO it's too early to compare and cherry-picked proofs both from Waymo and Tesla are not really representative.
- ra7 5y ago
  Waymo has published detailed safety performance data of their Arizona operations: https://waymo.com/safety/performance-data https://waymo.com/safety/performance-data You can read their other safety whitepapers in https://waymo.com/safety https://waymo.com/safety
  - gibolt 5y ago
    Arizona roads are also mapped to extreme precision, have very wide lanes, and are optimized for cars. Waymo has prioritized low intervention by being overly cautious and avoiding hard maneuvers (like many left turns). That doesn't work when they scale up to any other set of normal roads, especially as density and complexity increases.
    - ra7 5y ago
      They don't avoid left turns. There are plenty of videos from Chandler, AZ of Waymo performing unprotected left turns perfectly fine. They will always map roads to precision, whether it's Arizona or San Francisco. Why is that a problem? You should either look at their CA disengagement reports over the years or wait until they roll out a service in SF (where they've been testing heavily). That will show how safe they are in dense environments.
      - olau 5y ago
        From what I gather, they manually mark sections as hard when the cars get stuck there, e.g. due to road work, and then their routing system chooses another route, e.g. one that avoids the left turn. The video with the Waymo car getting stuck and taking off from the rescue team had an example of this. I guess it makes perfect sense from a engineering perspective.
jfim 5y ago
One of the gains from using lidar is also that it's a different sensor altogether from cameras, with different failure modes. For example, cameras are sensitive to glare from reflections (sun near sunset or reflecting on metallic objects) and oncoming traffic at night. Lidars operate on a different narrower wavelength and are unlikely to be affected by that, although they might struggle with objects that have low reflectivity at a long distance. The fact that these sensors are different means that the intersection where a dangerous situation would not be detected by either sensor is much smaller than any sensor individually. In any case, once AVs are deployed at scale, if it becomes apparent that some sensors can be removed or replaced by something else, then they will be if there's a case for it.

ra7 5y ago

> Prediction: Tesla will be the last of all major auto manufacturers to get to L5 autonomy. Tesla is also the only company to claim to target L5 autonomy. Everyone else, including Waymo, is strictly targeting L4 and say L5 autonomy is not possible or realistic. L5 is a pipe dream.

liuliu 5y ago

You get sparse point cloud from LIDAR sensors, not accurate 3D maps. This is the main reason why some people think LIDAR may not work well (mostly, only comma.ai and Tesla folks). Vision can also get you 3D maps, either in active manner (IR floodlight or structured lighting) or not. I will reserve my judgement until see more from either side.

bluepanda928752 5y ago
LIDAR is valuable as a safety feature, i.e. it can (unlike radars or cameras) reliably see (at least in clear weather) if there's anything in the car's path warranting evasion/braking maneuvers. In particular it's important that the LIDAR is dumb, i.e. its failure mores are predictable
- liuliu 5y ago
  LIDAR operates by timing how a photon reflects from a surface, it doesn't guarantee see everything. As you stated already, it cannot see in snow or rain. I am actually on sensor-fusion side, but I don't think we should jump to the conclusion LIDAR is the best 3D mapping method. For one example, a truck has a breaking distance of 600ft, either Velodyne or OS1 LIDAR has range less than that.
  - bluepanda928752 5y ago
    I agree about LIDARs not being the best general 3D mapping method, my point was mostly about using it as a dumb physics-based safety system. For autonomous trucks, limited LIDAR range could be mitigated by reducing speed accordingly and/or employing more powerful (or SWIR) LIDARs since trucks are more expensive and there could be a bigger budget for sensors
ra7 5y ago
> You get sparse point cloud from LIDAR sensors Take a look at how dense the point cloud from Waymo's 5th gen LIDAR is: https://www.youtube.com/watch?v=COgEQuqTAug&t=11599s https://www.youtube.com/watch?v=COgEQuqTAug&t=11599s. They just talked about this a few days ago.

etrautmann 5y ago

The vision-only strategy only made a shred of sense before high performance / low cost LIDARs like the Ouster OS1 became available. Now it’s an indefensible position on safety or economic grounds.

pokerhobo 5y ago
LIDAR in a drizzly city like Seattle is of minimal value so the system has to fallback to vision only anyways
- rpmisms 5y ago
  It's an anti-fragile approach, a term you'll recognize if you know of Nassim Taleb and his work. I think it will win in the long run, because not requiring HD Maps or specialized sensors is an advantage, and even if it takes more resources to make it work initially, it will save billions in the future, assuming, of course, it ever ships. Obviously, pure vision is a viable system, as it's what we as humans use. The question remains as to whether or not it will be comparable to more precise LiDAR based systems in the near future.
  - etrautmann 5y ago
    This is not a good argument overall for several reasons. First, we should aim to greatly exceed human performance on safety. Second, whatever work goes into making the algorithms work well using vision can just as easily still inform a system that fuses that data with LIDAR for enhanced situational awareness and safety.

tokipin 5y ago

I predict the opposite. Tesla sold half a million cars last year and will sell nearly one million this year. The data they have access to is increasing by orders of magnitude. I bet there is a point, let's say 20 million cars total, where they can pull so much high quality data that they will be able to surpass lidar capabilities for the purposes of self driving. The lidar/no lidar discussion is a fun one because people have different ideas about how the world works. Personally I think LiDAR is the modern version of expert systems. It appeals to a logical/geometric intuition but the approach is brittle to real world contact, especially when paired with HD maps which are a great way to drive yourself into a local maximum.

01100011 5y ago
The fallacy here is that the scale of the neural network used by Tesla is sufficient to capture the problem of driving given enough training. There is no guarantee that a reasonably priced neural network can encompass the task of driving. Having training data beyond a certain point is overrated, and Tesla's advantage in gathering it is overstated. Other companies are capturing this data as well. Is there any indication that the data Tesla is collecting is of a higher value, or is it just more bytes?
- daveguy 5y ago
  I'd say the data that Tesla collects is of lower value, because it doesn't have sensor info from a different modality. Other companies are getting a good reference to ground truth for both camera to lidar and lidar to camera. I don't know how much more valuable accurate distance sensing over a 3d field is compared to not having it, but I so know it's more valuable. It may be valuable enough to require a few petaflops less computing power.
- aimkey 5y ago
  It seems as if the people gobbling up the "Tesla has the data! Autopilot will keep getting better!" line have never trained a neural network in their life. Models converge. Loss stops decreasing, regardless of more incoming data. Extreme manual data cleaning effort becomes required to prevent overfitting. Model architecture has to change and hyper parameters have to be tweaked. Then you're back at square one as far as testing goes if you change any of those things. The notion that Tesla's model HAS to keep improving simply because they will be able to pile on more (unlabeled!) data is laughably false. And, in fact, quite insulting to the intelligence of even the most casual ML engineers.

codeulike 5y ago

Innovation is a gamble. You're not wrong to point out they might fail. The likeliness of failure is what makes it worth the gamble of trying.

bobsomers 5y ago
Generally this is true, but in the safety critical domain you don't gamble. You do your homework and make sure you aren't exposing your users to unnecessary levels of risk. If Tesla was developing their system with trained safety drivers or on closed courses, I think they would have higher moral ground to gamble here. But placing the untrained public behind the wheel of alpha quality software is unethical IMHO. There ways to develop autonomous software that are significantly less risky, and the only reason Tesla is doing it this way is for marketing/PR purposes as far as I can tell.

dheera 5y ago

My prediction is that Tesla will eventually use LIDAR despite whatever they are saying now. Right now their profit model is selling cars, not autonomy, so everything is optimized for that, including the decision to not use LIDAR.

sadfasf122 5y ago
lol what? did you watch the video at all
- dheera 5y ago
  Well yes, but I think they'll be singing a different tune in 5 years, and even more so when more Tesla cars actually start driving in weather conditions less favorable than Palo Alto and Austin. At some point when LIDAR is cheap enough there is no reason for Elon Musk to not give in and use them. Right now he's constraining the problem to the cost of the car.
  - minhazm 5y ago
    LiDAR is not better in worse weather conditions. LiDAR performance degrades in rain and snow. That’s where something like radar is better. The Tesla get is that humans drive with vision so they should be able to as well. Also every other self driving solution must solve the vision problem also in order to be successful. LiDAR doesn’t tell you that it’s a bag on the road vs a raccoon. So the question is that once you solve the vision problem, do you still need LiDAR for any meaningful impact?
    - dheera 5y ago
      > LiDAR doesn’t tell you that it’s a bag on the road vs a raccoon I think that's largely an issue with the early LIDAR devices today, but not necessarily what may be to come. There's something to be said about measuring actual data with solid physics vs. inferring distances with billions of operations on RGB data. If you were landing a commercial aircraft in fog, you most certainly don't rely on your eyes to do most of it, but it is in fact possible to do safely precisely because we do have good sensors on them. I fully agree with leveraging the scale and maturity of RGB sensors for cars today, the talk is spot on about that, but that's (a) circling back to the fact that Tesla needs to sell cars now not next year and (b) not a good case against use of LIDAR in the future.

cs702 5y ago

I strongly disagree. By all measures I've seen (including a couple of slides in the OP's video), Tesla's self-driving is far safer than human driving: the number of accidents and deaths per mile driven are something like an order of magnitude lower (i.e., around 10x safer). I mean, the machine never gets distracted, tired, sleepy, emotional, drunk, etc., so it is a LOT LESS likely to crash on boring, monotonous road segments than most people -- who do get distracted, tired, sleepy, etc. Not only that, but people make really scary mistakes in routine circumstances. The video shows several examples of human drivers hitting the accelerator when they actually meant to hit the brake! The criticism of autopilot is really about it getting tripped-up in response to statistically rare, unusual circumstances, i.e., edge cases. Karpathy et al are working on getting better at those, bringing the rate of situations that surprise autopilot closer and closer to 0%, even if it can never be achieved -- there will be always be surprises. Personally, I would rather take a tiny risk of crash on rare, once-in-a-million-miles events with autopilot driving than a ~1% risk of crash per 1000 to 2000 miles with everyday human driving. Prediction: Tesla will be the first of all major automakers to get to level 4 and 5 autonomy.

AndrewKemendo 5y ago
>Tesla's self-driving is far safer than human driving: the number of accidents and deaths per mile driven are something like an order of magnitude lower Still not close to good enough for people to accept: "Participants from both countries required Self Driving Vehicles to be 4-5 times as safe as Human Driven Vehicles" [0] 0: https://pubmed.ncbi.nlm.nih.gov/32202821/ https://pubmed.ncbi.nlm.nih.gov/32202821/
- deleted 5y ago
  [deleted]
- minhazm 5y ago
  It’s already about 10x. But it is a bit tricky since they don’t break the numbers out by highway and local driving. Also their active safety features outside of AP also improve safety. So does it need to be 4-5x better than an already improved system that has AEB and lane departure avoidance and other safety features? https://www.tesla.com/VehicleSafetyReport https://www.tesla.com/VehicleSafetyReport > In the 1st quarter, we registered one accident for every 4.19 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot but with our active safety features, we registered one accident for every 2.05 million miles driven. For those driving without Autopilot and without our active safety features, we registered one accident for every 978 thousand miles driven. By comparison, NHTSA’s most recent data shows that in the United States there is an automobile crash every 484,000 miles.

paxys 5y ago

> Prediction: Tesla will be the last of all major auto manufacturers to get to L5 autonomy That doesn't mean much considering no company is getting to L5 autonomy likely in our lifetime and possibly beyond.

vgchh 5y ago

Ultimately the proof is in the pudding. With Tesla FSD you can drive the highways from New York to Boston without any issues. I am sure there are many more routes across the country that can also be driven like that. Definitely not L5, but it works. Yet to see that from any other automaker, LIDAR or not. As far as I am concerned, great job Tesla! Keep it up, I am sure you will work through more tougher problems.

kemiller 5y ago

Waymo etc do not use LIDAR for object sensing, only for positioning. LIDAR sucks for object sensing because it gives you no information about whether it's a plastic bag or a person — you still need vision for that. Even if you just err on the safe side and brake, that itself can cause an accident unecessarily.

jowday 5y ago
You're wrong on that. LIDAR is used for object detection in every self driving car company that uses LIDAR. There's tons of research on it. Previous generations of LIDAR were not great at classifying small objects and road debris but it works great for detecting cars, pedestrians, etc. https://paperswithcode.com/task/3d-object-detection https://paperswithcode.com/task/3d-object-detection Next-gen LIDAR has great density, and I bet it would do a decent job differentiating between a plastic bag or a rock in the middle of the road. In addition to depth LIDAR also returns intensity and several other metrics, which can be used as input to an ML model. It's why you can read the lettering on the side of the semi truck in this video of Waymo's next-gen LIDAR. https://youtu.be/COgEQuqTAug?t=11601 https://youtu.be/COgEQuqTAug?t=11601

ec109685 5y ago

Why do you think the last 1% is dependent on LIDAR versus any of the other multitude of gaps between today’s autonomy and L5? If the only way it becomes practical to achieve L5 is to use LIDAR, Tesla can obviously add it. But if they waited until LIDAR was cheap and practical, they still wouldn’t be shipping any hardware doing autonomy today, and not collecting the data needed to train their models and delivering value today. Also, with vision based systems, it operates in somewhat an intuitive fashion given we have eyes too.

likearocket 5y ago

No one is going to get to a true L5 for a long time. That is totally irrelevant. It's a war of attrition. Whoever can monetize L3/L4 and can scale without any vehicle upgrade cost is going to win. It's pretty obvious lidar is very very silly since it doesn't scale. It is also pretty easy to see that Tesla doesn't have to hit L5 to have won autonomy. It just has to successfully monetize L3/L4.

jsight 5y ago

> solving the last 1% of this using only vision likely requires a general artificial intelligence That is likely close to true with lidar as well. See also some of Waymo's recent struggles in unexpected construction zones. Maybe lidar helps in getting there, but I'm afraid they all hit a pretty tough ceiling without this.

ra7 5y ago
> That is likely close to true with lidar as well. See also some of Waymo's recent struggles in unexpected construction zones. That example has nothing to do with lidar, but with planning. The car was able to detect the construction zone just fine.
- jsight 5y ago
  That's what I said.