7 ms·

So Lidar is getting cheaper, but the advantage of Lidar (Vs. optical) may be diminishing. Lidar worked really well because it was computationally impractical t

So Lidar is getting cheaper, but the advantage of Lidar (Vs. optical) may be diminishing. Lidar worked really well because it was computationally impractical to process visual images in near real-time onboard. Lidar simplifies the informational inflow, which reduces computational cost. But the computational cost and practicality of processing optical data onboard has changed radically, it is now practical and affordable (both due to substantially improved software and new/cheaper hardware). Keep in mind humans use "optical sensors with parallax" (i.e. our eyeballs). Cars are already optimised around that assumption (e.g. headlines to improve visibility in the visible light spectrum). Lidar still has advantages, but also a bunch of drawbacks (like dispersion and sunlight disruption), optical sensors do too (e.g. glare) but they're a lot more intuitive for humans because we share them. I guess what I am saying is: Are you betting on better sensor tech (Lidar) or better computational tech (Optical)? I think Lidar will hit a ceiling after the "easy wins" have been consumed (and we're approaching that point), whereas optical has no real ceiling (even over and above human's innate abilities). With optical you can understand the world as humans visibly see it, Lidar sees the world fundamentally differently, seeing both less (bad) and more (good). Obviously it is a somewhat false choice, but if people are investing dollars into development of both techs it is a choice that matters.

Infinitesimus 7y ago
> ... optical sensors do too (e.g. glare) but they're a lot more intuitive for humans because we share them That's an important point. The long term safer option will be to use both types of sensors before together, that can provide a more holistic picture of the scene around the car (much like we use radar for adaptive cruise control which would be much harder with just cameras ... I think). I hope we keep investing in making all these sensors cheaper and good enough for 30k cars to start packing them
- UI_at_80x24 7y ago
  Jumping on the 'agree' bandwagon. >The long term safer option will be to use both types of sensors before together This 100% is the real target but I'd like to add to it. Camera-based systems rely on ambient photons arranged in a pre-determinded configuration hitting a sensor. Lidar-based systems rely on reflected photons hitting a sensor. What I would prefer to see is a LiDAR/RADAR combination. When both systems rely on "photons" you suffer the same inherited weaknesses of both. i.e. fog But split it up and you can be SAFER.
  - magduf 7y ago
    >What I would prefer to see is a LiDAR/RADAR combination. When both systems rely on "photons" you suffer the same inherited weaknesses of both. i.e. fog Fog isn't the only problem with relying on ambient photos. "Nighttime" is a really big problem here. I really don't get this idea that optical sensors and processing are the solution for autonomous navigation. I propose this simple test for the pro-camera people: get in your car at 1AM, drive to a very rural place with a windy road, then turn off your headlights, and try driving on that road at 55mph. If you survive, come back and tell us how safe you think that exercise was. LiDAR doesn't have this problem, because it generates its own light source (a laser), and doesn't need overly-bright headlights to turn darkness into daytime, with all the problems that's now causing (light pollution, vision problems in people who drive at night from all the glare, etc.).
    - fennecfoxen 7y ago
      Why are we turning off our headlights in this exercise?
      - magduf 7y ago
        To illustrate the problems with optical sensors. What happens when the headlights fail, or if one of them fails, or if a bunch of snow or ice builds up in front of them? I also addressed some of the problems with relying on headlights at the end of my comment: to get better visibility, you need more light. This doesn't come without costs to the environment and to other drivers (like those who don't have driverless cars yet); in fact, too-bright lights are making it unsafe to drive at night now. As an aside, I think it should be illegal for anyone to have a car with non-halogen headlights without automatic high-beam dimming. Xenon and LED lights are great for illuminating dark roadways, but they're horrible when they on the car coming towards you, and that driver is too stupid or careless to dim their high beams. Temporary blindness is the result of this.
        fennecfoxen 7y ago
        > What happens when the headlights fail, or if one of them fails, or if a bunch of snow or ice builds up in front of them? What's expected of a normal driver when the headlights fail? Turning on the hazard blinkers, slowing down carefully, and finding their way to the shoulder? There's also a reason there are two headlights.
KaiserPro 7y ago
If we want to be safer, then we need to blend sensors. We do this as humans. We use our ears for sound and inertial detection. Lidar, Radar, colour and near infrared optical all have a place in autonomous driving. I dont think we are really anywhere near optical working reliably as a single source. I am immediately suspicious of anyone who suggests otherwise, because they either believe to much in the state of AI, or haven't thought enough about life critical systems to be let near anything autonomous. Worse still, they may be out to make a quick buck. The thing that lidar has is speed and accuracy. 100hz update rates, and 1/100th second latency is not unthinkable. With optical systems you're lucky to get 25hz with sub 100ms latency. Thats just for depth estimation using semantics, you still need to feed it into your driving model. Lidar works way better at night, it works a tonne better with unknown objects. However, like optical its shit in rain/snow. Hence why decent resolution radar will be needed as well.
- ganeshkrishnan 7y ago
  >If we want to be safer, then we need to blend sensors. isnt that what a kalman filter does? I thought most autonomous cars have sensor fusion to blend all sensors
  - CardenB 7y ago
    “Sensor fusion” isn’t just a thing, it’s an entire area of study. Kalman filtering is one technique in the umbrella of sensor fusion. There is a lot of room for improvement here.
  - teraflop 7y ago
    A Kalman filter solves the specific problem of estimating a finite-dimensional state vector from uncertain measurements, where the measurement errors and their correlations follow (or can be approximated as) a multivariate Gaussian. That works quite well for, say, combining GPS (which has short-term noise) with inertial/odometry measurements (which suffer from long-term drift) to determine your vehicle's position, orientation, and velocity in 3D space (expressible as a 9-dimensional state vector). But it's not directly applicable to problems like combining map data with LIDAR and vision to generate a representation of your surroundings.
andbberger 7y ago
It's not just the computational cost of real-time visual inference, there are some fundamental hurdles to be overcome we've attained truly human level computer vision. It's hard to imagine a pure vision system ever really matching the performance of a fused sensor system. Why infer when you can measure?
jmole 7y ago
we use optical sensors with parallax, yes, but they also have 6-dof motion capabilities. Cameras on a car are stuck in one place. Take some time and notice how often your head moves while driving. Even just the small motion of our eyes (rotation during fixation and tracking) adds spatial information to the visual stream that cameras simply can't achieve. A lot of the parallax information we infer is not because we have two eyes, but because we're able to move our eyes/head around, and "fill in the gaps" in a bayesian sort of way.
- SomewhatLikely 7y ago
  Cars can be driven remotely using only cameras, so the motion is likely not very critical. https://youtu.be/9sgetWQGYxY https://youtu.be/9sgetWQGYxY
- lazyjones 7y ago
  > Cameras on a car are stuck in one place. Take some time and notice how often your head moves while driving. That's why multiple cameras are used and positioned farther apart than you could move your head while inside the car.
  - threeseed 7y ago
    Those cameras are to provide a 360 view. There aren't enough of them to do a stereoscope view in any direction.
    - lmm 7y ago
      If true, that's easy to fix.
    - 8note 7y ago
      there's other ways of getting a stereoscopic view. comparing multiple images from the same camera at closeish points in time while in motion for instance.
    - jessaustin 7y ago
      I have only one eye, yet I've had no trouble obtaining a license and driving. Except in degenerate activities like ping-pong, "stereoscopic" viewing is not necessary.
threeseed 7y ago
Trying to emulate human eyesight in self driving cars is hilariously misguided. 1. We have a computer behind our eyes so advanced that we may never be able to come close to replicating it. It is capable of identifying, tracking and predicting the behaviour of multiple objects in real-time even in reduced visibility and can infer new objects without training e.g. a green firetruck or a RV with a satellite dish. 2. Our eyes are connected to a very adaptive and movable object i.e. our head. In order to perceive depth and identity objects e.g. an actual person versus a photo of a person we continuously move our head around in multiple dimensions. A car can't do this. I would refer everyone to the countless examples of Tesla's Autopilot recognising humans in bus signs, sides of trucks etc and attempting to do auto-avoidance. That is an unsolvable problem with only optical cameras.
- ebg13 7y ago
  > continuously move our head around in multiple dimensions. A car can't do this Doesn't isn't can't. There's no reason why car-mounted hardware can't move just as much as eyes do.
  - zdragnar 7y ago
    Metal fatigue (and other wear and tear) might be a reason; our muscles are self healing and regenerating. It would likely be cheaper and less error prone to simply have more sensors, not moving at all or with much narrower ranges of motion to simulate depth perception. Training a computer to understand a simulacrum is an entirely different challenge, I think.
  - threeseed 7y ago
    Of course cars can move their dozen or so cameras hundreds of times a second. But good luck getting a DNN to process all of that as well as the system handling tasks like identifying traffic lights, pedestrians. Head of Tesla AI has already stated that even with the new Nvidia hardware they struggle to meet the computational requirements.
    - ebg13 7y ago
NotSammyHagar 7y ago
That was an interesting observation, that lidar might actually be unnecessary by use of visual image processing. I always thought Tesla made a terrible choice in precluding (expensive) lidar and said if humans can drive safely without lidar, just using visual processing, so should cars (eventually). Surprising but I guess it may turn out that way.
trianglem 7y ago
I have serious doubts visual processing under all conditions could ever be better than a combination of both sensors.
mrfusion 7y ago
Maybe lidar could be great for training optical?
m463 7y ago
This will only augment current optical technology, not replace it.
deleted 7y ago
[deleted]
choppaface 7y ago
Lidar worked well (and continues to do so) because it’s the most accurate proximity sensor today for cars. If you don’t want your car to hit anything, you need Lidar. Radar and vision-based stereo are useful too, but 10%+ distance error and overall false negatives make that combo pretty unsafe (especially off the freeway). Computational cost is a far smaller factor than safety in the value of a demo. Cheap lidar (and probably radar) will earn a permanent place on ADAS of the future if only because it works much better than vision at night. You’ll see them together because they have complimentary failure modes and the denser lidar point cloud is worth a $100 add-on. While many of the autonomy efforts are still struggling with basic detection, the major challenge facing the leaders is in forecasting future motion, especially complex motion. The new cheap sensors are a boon but not a game changer.

Infinitesimus 7y ago

> ... optical sensors do too (e.g. glare) but they're a lot more intuitive for humans because we share them That's an important point. The long term safer option will be to use both types of sensors before together, that can provide a more holistic picture of the scene around the car (much like we use radar for adaptive cruise control which would be much harder with just cameras ... I think). I hope we keep investing in making all these sensors cheaper and good enough for 30k cars to start packing them

UI_at_80x24 7y ago
Jumping on the 'agree' bandwagon. >The long term safer option will be to use both types of sensors before together This 100% is the real target but I'd like to add to it. Camera-based systems rely on ambient photons arranged in a pre-determinded configuration hitting a sensor. Lidar-based systems rely on reflected photons hitting a sensor. What I would prefer to see is a LiDAR/RADAR combination. When both systems rely on "photons" you suffer the same inherited weaknesses of both. i.e. fog But split it up and you can be SAFER.
- magduf 7y ago
  >What I would prefer to see is a LiDAR/RADAR combination. When both systems rely on "photons" you suffer the same inherited weaknesses of both. i.e. fog Fog isn't the only problem with relying on ambient photos. "Nighttime" is a really big problem here. I really don't get this idea that optical sensors and processing are the solution for autonomous navigation. I propose this simple test for the pro-camera people: get in your car at 1AM, drive to a very rural place with a windy road, then turn off your headlights, and try driving on that road at 55mph. If you survive, come back and tell us how safe you think that exercise was. LiDAR doesn't have this problem, because it generates its own light source (a laser), and doesn't need overly-bright headlights to turn darkness into daytime, with all the problems that's now causing (light pollution, vision problems in people who drive at night from all the glare, etc.).
  - fennecfoxen 7y ago
    Why are we turning off our headlights in this exercise?
    - magduf 7y ago
      To illustrate the problems with optical sensors. What happens when the headlights fail, or if one of them fails, or if a bunch of snow or ice builds up in front of them? I also addressed some of the problems with relying on headlights at the end of my comment: to get better visibility, you need more light. This doesn't come without costs to the environment and to other drivers (like those who don't have driverless cars yet); in fact, too-bright lights are making it unsafe to drive at night now. As an aside, I think it should be illegal for anyone to have a car with non-halogen headlights without automatic high-beam dimming. Xenon and LED lights are great for illuminating dark roadways, but they're horrible when they on the car coming towards you, and that driver is too stupid or careless to dim their high beams. Temporary blindness is the result of this.
      - fennecfoxen 7y ago
        > What happens when the headlights fail, or if one of them fails, or if a bunch of snow or ice builds up in front of them? What's expected of a normal driver when the headlights fail? Turning on the hazard blinkers, slowing down carefully, and finding their way to the shoulder? There's also a reason there are two headlights.

KaiserPro 7y ago

If we want to be safer, then we need to blend sensors. We do this as humans. We use our ears for sound and inertial detection. Lidar, Radar, colour and near infrared optical all have a place in autonomous driving. I dont think we are really anywhere near optical working reliably as a single source. I am immediately suspicious of anyone who suggests otherwise, because they either believe to much in the state of AI, or haven't thought enough about life critical systems to be let near anything autonomous. Worse still, they may be out to make a quick buck. The thing that lidar has is speed and accuracy. 100hz update rates, and 1/100th second latency is not unthinkable. With optical systems you're lucky to get 25hz with sub 100ms latency. Thats just for depth estimation using semantics, you still need to feed it into your driving model. Lidar works way better at night, it works a tonne better with unknown objects. However, like optical its shit in rain/snow. Hence why decent resolution radar will be needed as well.

ganeshkrishnan 7y ago
>If we want to be safer, then we need to blend sensors. isnt that what a kalman filter does? I thought most autonomous cars have sensor fusion to blend all sensors
- CardenB 7y ago
  “Sensor fusion” isn’t just a thing, it’s an entire area of study. Kalman filtering is one technique in the umbrella of sensor fusion. There is a lot of room for improvement here.
- teraflop 7y ago
  A Kalman filter solves the specific problem of estimating a finite-dimensional state vector from uncertain measurements, where the measurement errors and their correlations follow (or can be approximated as) a multivariate Gaussian. That works quite well for, say, combining GPS (which has short-term noise) with inertial/odometry measurements (which suffer from long-term drift) to determine your vehicle's position, orientation, and velocity in 3D space (expressible as a 9-dimensional state vector). But it's not directly applicable to problems like combining map data with LIDAR and vision to generate a representation of your surroundings.

andbberger 7y ago

It's not just the computational cost of real-time visual inference, there are some fundamental hurdles to be overcome we've attained truly human level computer vision. It's hard to imagine a pure vision system ever really matching the performance of a fused sensor system. Why infer when you can measure?

jmole 7y ago

we use optical sensors with parallax, yes, but they also have 6-dof motion capabilities. Cameras on a car are stuck in one place. Take some time and notice how often your head moves while driving. Even just the small motion of our eyes (rotation during fixation and tracking) adds spatial information to the visual stream that cameras simply can't achieve. A lot of the parallax information we infer is not because we have two eyes, but because we're able to move our eyes/head around, and "fill in the gaps" in a bayesian sort of way.

SomewhatLikely 7y ago
Cars can be driven remotely using only cameras, so the motion is likely not very critical. https://youtu.be/9sgetWQGYxY https://youtu.be/9sgetWQGYxY
lazyjones 7y ago
> Cameras on a car are stuck in one place. Take some time and notice how often your head moves while driving. That's why multiple cameras are used and positioned farther apart than you could move your head while inside the car.
- threeseed 7y ago
  Those cameras are to provide a 360 view. There aren't enough of them to do a stereoscope view in any direction.
  - lmm 7y ago
    If true, that's easy to fix.
  - 8note 7y ago
    there's other ways of getting a stereoscopic view. comparing multiple images from the same camera at closeish points in time while in motion for instance.
  - jessaustin 7y ago
    I have only one eye, yet I've had no trouble obtaining a license and driving. Except in degenerate activities like ping-pong, "stereoscopic" viewing is not necessary.

threeseed 7y ago

Trying to emulate human eyesight in self driving cars is hilariously misguided. 1. We have a computer behind our eyes so advanced that we may never be able to come close to replicating it. It is capable of identifying, tracking and predicting the behaviour of multiple objects in real-time even in reduced visibility and can infer new objects without training e.g. a green firetruck or a RV with a satellite dish. 2. Our eyes are connected to a very adaptive and movable object i.e. our head. In order to perceive depth and identity objects e.g. an actual person versus a photo of a person we continuously move our head around in multiple dimensions. A car can't do this. I would refer everyone to the countless examples of Tesla's Autopilot recognising humans in bus signs, sides of trucks etc and attempting to do auto-avoidance. That is an unsolvable problem with only optical cameras.

ebg13 7y ago
> continuously move our head around in multiple dimensions. A car can't do this Doesn't isn't can't. There's no reason why car-mounted hardware can't move just as much as eyes do.
- zdragnar 7y ago
  Metal fatigue (and other wear and tear) might be a reason; our muscles are self healing and regenerating. It would likely be cheaper and less error prone to simply have more sensors, not moving at all or with much narrower ranges of motion to simulate depth perception. Training a computer to understand a simulacrum is an entirely different challenge, I think.
- threeseed 7y ago
  Of course cars can move their dozen or so cameras hundreds of times a second. But good luck getting a DNN to process all of that as well as the system handling tasks like identifying traffic lights, pedestrians. Head of Tesla AI has already stated that even with the new Nvidia hardware they struggle to meet the computational requirements.
  - ebg13 7y ago

NotSammyHagar 7y ago

That was an interesting observation, that lidar might actually be unnecessary by use of visual image processing. I always thought Tesla made a terrible choice in precluding (expensive) lidar and said if humans can drive safely without lidar, just using visual processing, so should cars (eventually). Surprising but I guess it may turn out that way.

trianglem 7y ago

I have serious doubts visual processing under all conditions could ever be better than a combination of both sensors.

mrfusion 7y ago

Maybe lidar could be great for training optical?

m463 7y ago

This will only augment current optical technology, not replace it.

deleted 7y ago

[deleted]

choppaface 7y ago

Lidar worked well (and continues to do so) because it’s the most accurate proximity sensor today for cars. If you don’t want your car to hit anything, you need Lidar. Radar and vision-based stereo are useful too, but 10%+ distance error and overall false negatives make that combo pretty unsafe (especially off the freeway). Computational cost is a far smaller factor than safety in the value of a demo. Cheap lidar (and probably radar) will earn a permanent place on ADAS of the future if only because it works much better than vision at night. You’ll see them together because they have complimentary failure modes and the denser lidar point cloud is worth a $100 add-on. While many of the autonomy efforts are still struggling with basic detection, the major challenge facing the leaders is in forecasting future motion, especially complex motion. The new cheap sensors are a boon but not a game changer.