5 ms·

They’re betting that they can use a massive feedback loop to train a set of neural networks to the point where they are as accurate as LiDAR without actually fi

They’re betting that they can use a massive feedback loop to train a set of neural networks to the point where they are as accurate as LiDAR without actually firing any lasers. Even if you believe this goal is possible to achieve at some point in the future, I think the argument falls apart when you consider that it will take years, probably decades, for a pure vision approach to catch up to where Waymo is today in terms of safety. (They have cameras too.) That Tesla can’t afford to fit expensive LiDAR sensors to all of the cars it sells is Tesla’s problem. Regulators won’t give a shit that pure vision is “better” in theory. They will simply compare Tesla’s crash rate in autonomous mode with that of Waymo and other AV operators, and act accordingly.

DSingularity 5y ago
Definitely. Especially with car companies like NIO strapping in LIDAR to their upcoming models.
bluepanda928752 5y ago
I understand why they made the "no-LIDAR" bet early when the LIDARs were completely unpractical for a production consumer car However, nowadays it starts to look that 100% reliable depth estimation from cameras might actually require a human-level AI to work and also solid-state LIDAR technology is becoming cheap enough and integrateable into normal cars, but Tesla can't really change their stance on this without admitting that FSD options they already sold would not actually become FSD within the lifetimes of these vehicles. I suspect this might also be the reason why Karpathy looks more and more nervous with each new talk
- anitil 5y ago
  Is it something that can be retrofitted on existing models?
  - bluepanda928752 5y ago
    LIDARs unlikely since they require a line of sight forwards, backwards and possibly on the sides of the vehicle
- donio 5y ago
  > FSD options they already sold would not actually become FSD within the lifetimes of these vehicles That's pretty much a given at this point but they will not admit it until a class-action lawsuit forces them to.
  - bluepanda928752 5y ago
    But we might actually get a functioning Mars base out of this since that's where Musk will be hiding when the FTC finally wakes up /s
- trhway 5y ago
  >100% reliable depth estimation from cameras might actually require a human-level AI to work you don't need 100%, and even humans are far from 100% (500Mpx resolution of our eyes allows to basically sheer brute force through it in many cases). The stereo setup provides great and fast estimation with several megapixel resolution with good fps (way better than lidar) for majority of situations. It is some share of the [part of the] scenes, and you really know it right then and there, where you need AI and/or very sophisticated compute heavy algorithms. So instead of throwing AI and the compute power at those parts, you just pull the points from the lidar (and even radar if the things are that bad) covering that segment. And that way, given a couple more iterations of sensors (from current 20Mpx+ to the hundreds Mpx) and compute, it will be doing even better than humans. Anybody not doing sensor fusion would be a loser though - just like going into a fist fight with one hand intentionally disabled.
practice9 5y ago
> I think the argument falls apart when you consider that it will take years, probably decades, for a pure vision approach to catch up to where Waymo is today in terms of safety. (They have cameras too.) On what set of metrics do you think Waymo is safer? IMO it's too early to compare and cherry-picked proofs both from Waymo and Tesla are not really representative.
- ra7 5y ago
  Waymo has published detailed safety performance data of their Arizona operations: https://waymo.com/safety/performance-data https://waymo.com/safety/performance-data You can read their other safety whitepapers in https://waymo.com/safety https://waymo.com/safety
  - gibolt 5y ago
    Arizona roads are also mapped to extreme precision, have very wide lanes, and are optimized for cars. Waymo has prioritized low intervention by being overly cautious and avoiding hard maneuvers (like many left turns). That doesn't work when they scale up to any other set of normal roads, especially as density and complexity increases.
    - ra7 5y ago
      They don't avoid left turns. There are plenty of videos from Chandler, AZ of Waymo performing unprotected left turns perfectly fine. They will always map roads to precision, whether it's Arizona or San Francisco. Why is that a problem? You should either look at their CA disengagement reports over the years or wait until they roll out a service in SF (where they've been testing heavily). That will show how safe they are in dense environments.
      - olau 5y ago
        From what I gather, they manually mark sections as hard when the cars get stuck there, e.g. due to road work, and then their routing system chooses another route, e.g. one that avoids the left turn. The video with the Waymo car getting stuck and taking off from the rescue team had an example of this. I guess it makes perfect sense from a engineering perspective.
jfim 5y ago
One of the gains from using lidar is also that it's a different sensor altogether from cameras, with different failure modes. For example, cameras are sensitive to glare from reflections (sun near sunset or reflecting on metallic objects) and oncoming traffic at night. Lidars operate on a different narrower wavelength and are unlikely to be affected by that, although they might struggle with objects that have low reflectivity at a long distance. The fact that these sensors are different means that the intersection where a dangerous situation would not be detected by either sensor is much smaller than any sensor individually. In any case, once AVs are deployed at scale, if it becomes apparent that some sensors can be removed or replaced by something else, then they will be if there's a case for it.

DSingularity 5y ago

Definitely. Especially with car companies like NIO strapping in LIDAR to their upcoming models.

bluepanda928752 5y ago

I understand why they made the "no-LIDAR" bet early when the LIDARs were completely unpractical for a production consumer car However, nowadays it starts to look that 100% reliable depth estimation from cameras might actually require a human-level AI to work and also solid-state LIDAR technology is becoming cheap enough and integrateable into normal cars, but Tesla can't really change their stance on this without admitting that FSD options they already sold would not actually become FSD within the lifetimes of these vehicles. I suspect this might also be the reason why Karpathy looks more and more nervous with each new talk

anitil 5y ago
Is it something that can be retrofitted on existing models?
- bluepanda928752 5y ago
  LIDARs unlikely since they require a line of sight forwards, backwards and possibly on the sides of the vehicle
donio 5y ago
> FSD options they already sold would not actually become FSD within the lifetimes of these vehicles That's pretty much a given at this point but they will not admit it until a class-action lawsuit forces them to.
- bluepanda928752 5y ago
  But we might actually get a functioning Mars base out of this since that's where Musk will be hiding when the FTC finally wakes up /s
trhway 5y ago
>100% reliable depth estimation from cameras might actually require a human-level AI to work you don't need 100%, and even humans are far from 100% (500Mpx resolution of our eyes allows to basically sheer brute force through it in many cases). The stereo setup provides great and fast estimation with several megapixel resolution with good fps (way better than lidar) for majority of situations. It is some share of the [part of the] scenes, and you really know it right then and there, where you need AI and/or very sophisticated compute heavy algorithms. So instead of throwing AI and the compute power at those parts, you just pull the points from the lidar (and even radar if the things are that bad) covering that segment. And that way, given a couple more iterations of sensors (from current 20Mpx+ to the hundreds Mpx) and compute, it will be doing even better than humans. Anybody not doing sensor fusion would be a loser though - just like going into a fist fight with one hand intentionally disabled.

practice9 5y ago

> I think the argument falls apart when you consider that it will take years, probably decades, for a pure vision approach to catch up to where Waymo is today in terms of safety. (They have cameras too.) On what set of metrics do you think Waymo is safer? IMO it's too early to compare and cherry-picked proofs both from Waymo and Tesla are not really representative.

ra7 5y ago
Waymo has published detailed safety performance data of their Arizona operations: https://waymo.com/safety/performance-data https://waymo.com/safety/performance-data You can read their other safety whitepapers in https://waymo.com/safety https://waymo.com/safety
- gibolt 5y ago
  Arizona roads are also mapped to extreme precision, have very wide lanes, and are optimized for cars. Waymo has prioritized low intervention by being overly cautious and avoiding hard maneuvers (like many left turns). That doesn't work when they scale up to any other set of normal roads, especially as density and complexity increases.
  - ra7 5y ago
    They don't avoid left turns. There are plenty of videos from Chandler, AZ of Waymo performing unprotected left turns perfectly fine. They will always map roads to precision, whether it's Arizona or San Francisco. Why is that a problem? You should either look at their CA disengagement reports over the years or wait until they roll out a service in SF (where they've been testing heavily). That will show how safe they are in dense environments.
    - olau 5y ago
      From what I gather, they manually mark sections as hard when the cars get stuck there, e.g. due to road work, and then their routing system chooses another route, e.g. one that avoids the left turn. The video with the Waymo car getting stuck and taking off from the rescue team had an example of this. I guess it makes perfect sense from a engineering perspective.

jfim 5y ago

One of the gains from using lidar is also that it's a different sensor altogether from cameras, with different failure modes. For example, cameras are sensitive to glare from reflections (sun near sunset or reflecting on metallic objects) and oncoming traffic at night. Lidars operate on a different narrower wavelength and are unlikely to be affected by that, although they might struggle with objects that have low reflectivity at a long distance. The fact that these sensors are different means that the intersection where a dangerous situation would not be detected by either sensor is much smaller than any sensor individually. In any case, once AVs are deployed at scale, if it becomes apparent that some sensors can be removed or replaced by something else, then they will be if there's a case for it.