15 ms·

This is a newbie question, but why is the lidar/radar necessary? Is it solely because computers aren't yet able to extract as much information from video as hum

This is a newbie question, but why is the lidar/radar necessary? Is it solely because computers aren't yet able to extract as much information from video as humans can from sight? Theoretically, could self driving cars use just video with enough compute resources and the right ML algorithms?

MertsA 7y ago
There's a whole lot of challenging information that is completely natural and intuitive for a human to understand but fiendishly difficult for a ML algorithm to figure out. There's some cues that I'm sure we probably won't be able to use until we create a genuine artificial general intelligence. If you're driving and you see someone standing at a crosswalk it's intuitive just by looking at them if they're waiting for you to pass, about to walk out into the road, panhandling, etc. You can put yourself in their shoes and make a reasonable prediction as to what they are thinking about doing. On a previous HN thread there was a commenter that had a run in with a Waymo car while riding a bike. He was coming up to a 4 way stop, and yielding to the Waymo car. If he was balancing on his pedals, still stationary, the Waymo car would stop and not proceed through the intersection, apparently interpreting his pose as if he's about to proceed through the intersection itself. A human driver wouldn't have an issue with that, but you can imagine the training data would show that when a bicycle is stopped, the rider puts a foot on the ground. Generous helpings of lidar and radar to augment cameras is a crutch to help compensate for the lack of 500 million years of unsupervised learning that went into our visual cortex.
- mavhc 7y ago
  Lidar or radar won't help you with any of those situations though
  - MertsA 7y ago
    For those specific cases no, but it makes a great source to identify physical objects and their placement in the world. Obviously that can be done to a sufficient degree with just stereoscopic vision, but look at all of the autopilot fatalities from Tesla. All too often vision and radar data gets misinterpreted as "That must be an object to the side of the road" right up until it crashes into it. With lidar you can be sure you're not just looking at a radar reflection or getting the perspective wrong on a camera. Almost all of the large objects that you want to make sure you never hit will show up well with lidar. https://en.wikipedia.org/wiki/Tesla_Autopilot#Incidents https://en.wikipedia.org/wiki/Tesla_Autopilot#Incidents Just about every one of those fatalities can be summed up as "Tesla ignores large stationary object directly ahead". Lidar would have detected all of those objects and most likely prevented every one of those accidents. I think Tesla currently has the best vision and radar only system out there, so either the state of the art doesn't quite cut it yet without lidar, or there's a ML engineer at Tesla that really really hates fire trucks.
    - mavhc 7y ago
      Or Telsa doesn't actually provide Full Self Driving yet and people shouldn't be watching their phones. It is noticing stationary objects, because sometimes it breaks when the car approaches a bridge going overhead, which is also bad when the car too close behind doesn't. You have to ignore some objects in front of you (even ones heading directly towards you) because you're going round corners, so it's never cut and dried
- vbezhenar 7y ago
  It might look intuitive just by looking at them, but good drivers won't trust that intuition, because humans are very unpredictable species. So good drivers will slow down to have enough time to react if necessary. You should not trust "reasonable prediction" when you're talking about death danger.
  - Piskvorrr 7y ago
    Which means "do not drive, ever." The actual risk appetite for (human) driving is very different from the advertised one; the heated discussions on SDV just bring this to light.
deleted 7y ago
[deleted]
marcosdumay 7y ago
Computer vision is a very noisy application. Unless you have some extremely good software filters, that noise will reduce reliability, what in a car may end in disaster. Even humans are fooled by it once in a while. And even if Google has those extremely good filters, there is an entire different order of magnitude of testing until they can be sure. They are probably jut trying to get something out of the lab by using reliable sensors, and letting optimizations for later.
Grue3 7y ago
There's no practical reason to limit autonomous vehicles to human-visible signals. The point is to make a car that drives safer than a human. Since we can't make a car that's smarter than a human, we can at least even the odds and make it see much better.
- zaroth 7y ago
  Actually the reason is the most practical one of all; money. A system which costs too much to widely deploy and maintain will never see widespread deployment.
wefarrell 7y ago
Humans are a pretty poor baseline. A fully autonomous driving system that was only as good as a human would not gain much traction and the company would be on the hook for some serious legal damages if it became popular.
- nwsm 7y ago
  You make it sound like professional drivers are currently personally liable when they cause accidents. Is this true? (I have no idea)
UI_at_80x24 7y ago
Picture this scenario: While looking at a TV screen; what is the difference between a photograph of a car (on the screen) vs a real car (on the screen)? The only thing a camera can offer is input for pattern recognition. Lidar/Radar offers context. I'm not suggesting that all you need to fool computer assisted driving is a picture of an empty road taped to the front of the camera; but if the computer can't tell the difference between an optical illusion and reality then I think they need more data inputs. We don't need computers to be able to handle white-out blizzard driving conditions, but that 'rare' occurrence perfectly illustrates the singular limitation of a camera-based pattern-recognition system.
- jiofih 7y ago
  How do you think a person distinguishes something on a TV screen and the real thing? We don’t have any kind of 3D perception like Lidar.
IshKebab 7y ago
Yeah basically LIDAR is better than vision for working out the 3D structure of the world. You can do it from vision, but it's really really hard. We don't have a working driverless car with nice easy LIDAR data - it would be really silly to try and make one the hard way first, using only vision data (yes I know about Telsa).
sangnoir 7y ago
Why should we pursue biomimicry to that extent? My go to example is to compare aircraft to birds, so I could rephrase your question as why are the engine/fixed wings necessary? Is it solely because material engineering hasn't advanced yet to enable airplanes to flap their wings to stay airborne like birds? Human eyes and brains are completely different from solid state electronics with different constraints and advantages.
- 0xffff2 7y ago
  There is in fact a pretty long history of biomimicry in aviation though. I've met many aerospace engineers (including a former NASA chief scientist) who would take your question very seriously. The answer may well be "no, that isn't the only reason", but it's not trivial to arrive at that answer.
anthonybsd 7y ago
Currently it's very difficult to reason about object geometric dimensions / movement from video alone. We (humans) are decent at it and we still make mistakes. "The right ML algorithm" for this is basically a problem as hard as achieving singularity.
denkmoon 7y ago
Theoretically, since humans drive "ok" using just 2 eyes, you must be able to do it with two cameras and a sufficiently complex brain. Of course, why would you not avail yourself of more sensory information if you can, provided the cost is not overly burdensome.
ebg13 7y ago
It isn't necessary. As you suggest, humans have only normal eyes, not laser eyes, and we still manage to drive. Yes and no. It's also just nice to have more different kinds of data available. Just because humans don't have laser eyes doesn't mean we can't try to do better than that. Depth from stereo is not ML, but yes.
- StavrosK 7y ago
  It isn't "necessary" for small values of "necessary", ie if you have a human brain hooked up to human eyes. It is extremely necessary if you only have a computer and some primitive software, like we do now, as in "self-driving won't work as well otherwise".
- herogreen 7y ago
  Yet depth from stereo is, to some extent, AI because the matching of pixels that corresponds to the same physical point is not trivial and uses heuristics.
  - Animats 7y ago
    Depth from stereo gets worse at distance, while LIDAR and RADAR read depth directly.
- mlyle 7y ago
  > Depth from stereo is not ML, but yes. Tesla in particular, and others are moving towards getting depth from ML. Yes, you can do dumb coincidence-finding, but there's a lot of corner cases (leafy objects, specular reflections, etc) that screw this up. Humans don't just use coincidence finding, but use all kinds of other clues (size, texture gradients, monocular parallax, shadows, linear perspective, attenuation from haze, etc) to infer depth.
  - tyfon 7y ago
    Tesla is using data from the radar to annotate the images from the cameras with distance before running it through the ML training, but for driving it is using a combination of all (stereo vision, ML and radar) to produce the final number. The human neural network is of course fine tuned to this and my father who is blind on one eye will actually move his head sideways a bit to judge distance when he is driving :)
Daub 7y ago
A camera (or a binocular camera setup) could only ever return depth planes. Lidar returns true 3d forms. Also, depth plane evaluation from cameras is not 100% reliable. Things like reflections, glare and suchlike screw it up.
- IshKebab 7y ago
  This is nonsense. There is no difference between the data that binocular cameras return and LIDAR, other than that LIDAR is more reliable and accurate. Your distinction between "depth planes" and "true 3d forms" is gobbledygook.
  - Daub 7y ago
    Maybe I did not express myself accurately enough. I work in VFX, and my language is not always as precise as it could be. But certainly for us this is the difference in application. Using a two camera input, the best we could hope to get was a depth map. Typically, it is reliable for at most one or two reliable levels of depth, useful for foreground/background separation. Maybe also a crude depth map, useful for fog. With Lidar we could all that plus normal information (i.e. face orientation). However, I concede your point. The core difference is one of accuracy.
deleted 7y ago
[deleted]
sdan 7y ago
It’s all about redundancy. lidars do help though immensely if you integrate it in your ML algorithms currently
vkou 7y ago
> This is a newbie question, but why is the lidar/radar necessary? It's not, in theory. It is, in practice. > > Is it solely because computers aren't yet able to extract as much information from video as humans can from sight? Yes. Currently, you cannot, for any amount of money, buy a camera that has equivalent visual acuity to the human eye.
- nneonneo 7y ago
  I’m not convinced that the eye is conclusively better than any camera out there. The human brain is incredible at stitching together visual input into a coherent view of the world. The retina has high resolution but covers an incredibly small field of view; rapid eye movements make it possible to fake a larger effective field of view. And I’m not convinced that the retina’s resolution is actually better than that of high-end camera systems. The eye’s performance in the periphery is decidedly worse than that of a wide-angle camera. Dynamic range is a challenge for cameras, but we have high-dynamic-range imaging nowadays (both in software, i.e. exposure combining, and hardware). I don’t think the eye is significantly superior here. Low-light used to be a strength of the human visual system, but I think modern computational imaging systems have caught up (plus, human night vision was really never that good compared with other animals). So in short: I dispute the idea that cameras don’t have the acuity of the human visual system, nowadays. I’d like to know in which aspects you believe human eyes to still be superior, from an optical point of view. Obviously - the human brain and visual cortex is something that computers are nowhere close to.
  - vkou 7y ago
    > So in short: I dispute the idea that cameras don’t have the acuity of the human visual system, nowadays. I’d like to know in which aspects you believe human eyes to still be superior, from an optical point of view. Dynamic range. Your eyes can pick up subtle details in a scene with very bright lights, and very dark shadows. No video camera currently exists that can take a good video from inside a city, at night, with a starry night sky. Either the stars, or the lights, or the shadows are going to look like crap. Your eyes can trivially handle such a problem.
- 7y ago
sumgame 7y ago
It isn't necessary and the best example of this is Tesla's autopilot. It uses no Lidar component and Elon Musk's bet is that self-driving cars can just use just cameras, basic radar and ultrasonic sensors with enough compute resources and the right ML algorithms to perform better than a human. He's been a long time anti lidar proponent because of the costs involved and the aesthetics. He's also betting that the amount of data Tesla receives from its customers, and the neural net they have can achieve autonomous driving with its current hardware stack. “In my view, it’s a crutch that will drive companies to a local maximum that they will find very hard to get out of,” Musk said. He added, “Perhaps I am wrong, and I will look like a fool. But I am quite certain that I am not. "Despite being a fancy and expensive technology, LiDAR provides surprisingly little advantage over a combination of cameras and radar. Radar, for example, is much better in the rain and other limited visibility scenarios, because it is based on radio waves rather than light waves. Radio can penetrate through some objects and bounce back from others, thereby “seeing” the environment along a different dimension." an excerpt from a quora answer on why the Tesla stack could be better than Lidar: https://www.quora.com/Why-dont-Tesla-cars-use-Lidar-like-most-self-driving-cars-in-the-industry?share=1 https://www.quora.com/Why-dont-Tesla-cars-use-Lidar-like-mos... Here's a video of Tesla's autopilot perceives its environment : https://www.youtube.com/watch?v=fKXztwtXaGo https://www.youtube.com/watch?v=fKXztwtXaGo and here's a video of how Waymo perceives its environment: https://www.youtube.com/watch?v=OopTOjnD3qY https://www.youtube.com/watch?v=OopTOjnD3qY The question is if the Lidar adds incremental value or exponential value, and I think it's just incremental by looking at those videos.
- karlp 7y ago
  A few people died following that bet though
  - sumgame 7y ago
    I don't have a Tesla nor am I a fanboy. Just talking about the tech. This is biased information as there are way more Tesla's out there compared to Waymo's. Waymo has had accidents too https://www.wired.com/story/waymo-crash-self-driving-google-arizona/ https://www.wired.com/story/waymo-crash-self-driving-google-...
scarejunba 7y ago
Besides the other reasons it's also that the dynamic range of eyes are way higher than the cameras they can use here. And humans have access to mechanisms that allow them to handle washouts. They're just building in the equivalent of putting in your sunglasses or looking off to the side when the sun is in your eyes or readjusting your seat position so it isn't hitting your eyes.
summerlight 7y ago
Self-driving tech is still at an immature stage while the bar set by public is super high; even human-level self driving might be unacceptable for pervasive adoption. Anyway you can use both Lidar and camera. It's not too late to worry about this cost-performance trade-off after making it actually work.
Traster 7y ago
There are a lot of people here very confidently stating that LIDAR isn't necessary but seemingly forgetting to link to any actual evidence of fully autonomous vehicles that work without it. The issue is that human vision isn't just vision, it's vision linked with a ridiculous general intelligence that we're no where close to reproducing. So humans are great at categorising the objects they see, and reasoning and how they'll behave, which is why we don't generally need LIDAR. So it's just as much about how humans can reason from sight as it is about how good their sight actually is. It's very possible that by the time you've built a computer capable of the same sort of reasoning that humans do, you've actually solved, well, artifical general intelligence and you're no longer designing a car at all - because the AGI took that job ages ago.
- nikanj 7y ago
  According to your average SV pitch deck, we are 6-10 months away from reproducing general intelligence. Add in a sprinkling of Pascal's Wager, and your funding is almost guaranteed.
  - Piskvorrr 7y ago
    Which goes back all the way to the Dartmouth workshop, which hoped to get most of the work done in 6-8 weeks. That was in 1956. Is most of the work done? Well there you go. (Your hint that it's a good way to get paid for such work holds, of course, but the expectation that Problems will get Solved...unlikely)
- vbezhenar 7y ago
  I'm not sure that human uses that general intelligence when driving and when decisions must be made quickly. I think that brain uses shortcuts and does not really reason about things. And those shortcuts are likely work similar to trained neural networks. That's one reason why novice drivers have problems: they don't have those shortcuts in their brains yet, so they have to reason and that takes valuable time and attention. Though I think that robotic automobiles must perform better than humans, so equipping them with better sensors makes perfect sense, regardless of algorithms.
yogrish 7y ago
Even in broad day light and hith High Res videos and with ML algos, we cannot extract all information accurately that are needed. For example, at what velocity the other vehicle is moving and also at what distance we have an obstacle etc. is very difficult to extract from camera alone and impossible when it is night or extreme weather conditions. Hence we need Radar & Lidars also for ADAS & Self driving applications. Radar: It can accurately measure Distance and Velocity information of objects around ego vehicle and also can track objects. It works well in all weather conditions (Day/night/rain/fog etc). Lidar: Good distance measurement, Rich in data (3D Point cloud) for ML, OK in doing classification (pedestrain, bicyclist etc) even in nights. But expensive sensor.
sreekotay 7y ago
Lots of debate on the topic (clearly :)) But arguably very little progress (none?) in technology, automation, and industrialization has been through dogmatic replication of biological systems. Planes don't fly like birds...

MertsA 7y ago

There's a whole lot of challenging information that is completely natural and intuitive for a human to understand but fiendishly difficult for a ML algorithm to figure out. There's some cues that I'm sure we probably won't be able to use until we create a genuine artificial general intelligence. If you're driving and you see someone standing at a crosswalk it's intuitive just by looking at them if they're waiting for you to pass, about to walk out into the road, panhandling, etc. You can put yourself in their shoes and make a reasonable prediction as to what they are thinking about doing. On a previous HN thread there was a commenter that had a run in with a Waymo car while riding a bike. He was coming up to a 4 way stop, and yielding to the Waymo car. If he was balancing on his pedals, still stationary, the Waymo car would stop and not proceed through the intersection, apparently interpreting his pose as if he's about to proceed through the intersection itself. A human driver wouldn't have an issue with that, but you can imagine the training data would show that when a bicycle is stopped, the rider puts a foot on the ground. Generous helpings of lidar and radar to augment cameras is a crutch to help compensate for the lack of 500 million years of unsupervised learning that went into our visual cortex.

mavhc 7y ago
Lidar or radar won't help you with any of those situations though
- MertsA 7y ago
  For those specific cases no, but it makes a great source to identify physical objects and their placement in the world. Obviously that can be done to a sufficient degree with just stereoscopic vision, but look at all of the autopilot fatalities from Tesla. All too often vision and radar data gets misinterpreted as "That must be an object to the side of the road" right up until it crashes into it. With lidar you can be sure you're not just looking at a radar reflection or getting the perspective wrong on a camera. Almost all of the large objects that you want to make sure you never hit will show up well with lidar. https://en.wikipedia.org/wiki/Tesla_Autopilot#Incidents https://en.wikipedia.org/wiki/Tesla_Autopilot#Incidents Just about every one of those fatalities can be summed up as "Tesla ignores large stationary object directly ahead". Lidar would have detected all of those objects and most likely prevented every one of those accidents. I think Tesla currently has the best vision and radar only system out there, so either the state of the art doesn't quite cut it yet without lidar, or there's a ML engineer at Tesla that really really hates fire trucks.
  - mavhc 7y ago
    Or Telsa doesn't actually provide Full Self Driving yet and people shouldn't be watching their phones. It is noticing stationary objects, because sometimes it breaks when the car approaches a bridge going overhead, which is also bad when the car too close behind doesn't. You have to ignore some objects in front of you (even ones heading directly towards you) because you're going round corners, so it's never cut and dried
vbezhenar 7y ago
It might look intuitive just by looking at them, but good drivers won't trust that intuition, because humans are very unpredictable species. So good drivers will slow down to have enough time to react if necessary. You should not trust "reasonable prediction" when you're talking about death danger.
- Piskvorrr 7y ago
  Which means "do not drive, ever." The actual risk appetite for (human) driving is very different from the advertised one; the heated discussions on SDV just bring this to light.

deleted 7y ago

[deleted]

marcosdumay 7y ago

Computer vision is a very noisy application. Unless you have some extremely good software filters, that noise will reduce reliability, what in a car may end in disaster. Even humans are fooled by it once in a while. And even if Google has those extremely good filters, there is an entire different order of magnitude of testing until they can be sure. They are probably jut trying to get something out of the lab by using reliable sensors, and letting optimizations for later.

Grue3 7y ago

There's no practical reason to limit autonomous vehicles to human-visible signals. The point is to make a car that drives safer than a human. Since we can't make a car that's smarter than a human, we can at least even the odds and make it see much better.

zaroth 7y ago
Actually the reason is the most practical one of all; money. A system which costs too much to widely deploy and maintain will never see widespread deployment.

wefarrell 7y ago

Humans are a pretty poor baseline. A fully autonomous driving system that was only as good as a human would not gain much traction and the company would be on the hook for some serious legal damages if it became popular.

nwsm 7y ago
You make it sound like professional drivers are currently personally liable when they cause accidents. Is this true? (I have no idea)

UI_at_80x24 7y ago

Picture this scenario: While looking at a TV screen; what is the difference between a photograph of a car (on the screen) vs a real car (on the screen)? The only thing a camera can offer is input for pattern recognition. Lidar/Radar offers context. I'm not suggesting that all you need to fool computer assisted driving is a picture of an empty road taped to the front of the camera; but if the computer can't tell the difference between an optical illusion and reality then I think they need more data inputs. We don't need computers to be able to handle white-out blizzard driving conditions, but that 'rare' occurrence perfectly illustrates the singular limitation of a camera-based pattern-recognition system.

jiofih 7y ago
How do you think a person distinguishes something on a TV screen and the real thing? We don’t have any kind of 3D perception like Lidar.

IshKebab 7y ago

Yeah basically LIDAR is better than vision for working out the 3D structure of the world. You can do it from vision, but it's really really hard. We don't have a working driverless car with nice easy LIDAR data - it would be really silly to try and make one the hard way first, using only vision data (yes I know about Telsa).

sangnoir 7y ago

Why should we pursue biomimicry to that extent? My go to example is to compare aircraft to birds, so I could rephrase your question as why are the engine/fixed wings necessary? Is it solely because material engineering hasn't advanced yet to enable airplanes to flap their wings to stay airborne like birds? Human eyes and brains are completely different from solid state electronics with different constraints and advantages.

0xffff2 7y ago
There is in fact a pretty long history of biomimicry in aviation though. I've met many aerospace engineers (including a former NASA chief scientist) who would take your question very seriously. The answer may well be "no, that isn't the only reason", but it's not trivial to arrive at that answer.

anthonybsd 7y ago

Currently it's very difficult to reason about object geometric dimensions / movement from video alone. We (humans) are decent at it and we still make mistakes. "The right ML algorithm" for this is basically a problem as hard as achieving singularity.

denkmoon 7y ago

Theoretically, since humans drive "ok" using just 2 eyes, you must be able to do it with two cameras and a sufficiently complex brain. Of course, why would you not avail yourself of more sensory information if you can, provided the cost is not overly burdensome.

ebg13 7y ago

It isn't necessary. As you suggest, humans have only normal eyes, not laser eyes, and we still manage to drive. Yes and no. It's also just nice to have more different kinds of data available. Just because humans don't have laser eyes doesn't mean we can't try to do better than that. Depth from stereo is not ML, but yes.

StavrosK 7y ago
It isn't "necessary" for small values of "necessary", ie if you have a human brain hooked up to human eyes. It is extremely necessary if you only have a computer and some primitive software, like we do now, as in "self-driving won't work as well otherwise".
herogreen 7y ago
Yet depth from stereo is, to some extent, AI because the matching of pixels that corresponds to the same physical point is not trivial and uses heuristics.
- Animats 7y ago
  Depth from stereo gets worse at distance, while LIDAR and RADAR read depth directly.
mlyle 7y ago
> Depth from stereo is not ML, but yes. Tesla in particular, and others are moving towards getting depth from ML. Yes, you can do dumb coincidence-finding, but there's a lot of corner cases (leafy objects, specular reflections, etc) that screw this up. Humans don't just use coincidence finding, but use all kinds of other clues (size, texture gradients, monocular parallax, shadows, linear perspective, attenuation from haze, etc) to infer depth.
- tyfon 7y ago
  Tesla is using data from the radar to annotate the images from the cameras with distance before running it through the ML training, but for driving it is using a combination of all (stereo vision, ML and radar) to produce the final number. The human neural network is of course fine tuned to this and my father who is blind on one eye will actually move his head sideways a bit to judge distance when he is driving :)

Daub 7y ago

A camera (or a binocular camera setup) could only ever return depth planes. Lidar returns true 3d forms. Also, depth plane evaluation from cameras is not 100% reliable. Things like reflections, glare and suchlike screw it up.

IshKebab 7y ago
This is nonsense. There is no difference between the data that binocular cameras return and LIDAR, other than that LIDAR is more reliable and accurate. Your distinction between "depth planes" and "true 3d forms" is gobbledygook.
- Daub 7y ago
  Maybe I did not express myself accurately enough. I work in VFX, and my language is not always as precise as it could be. But certainly for us this is the difference in application. Using a two camera input, the best we could hope to get was a depth map. Typically, it is reliable for at most one or two reliable levels of depth, useful for foreground/background separation. Maybe also a crude depth map, useful for fog. With Lidar we could all that plus normal information (i.e. face orientation). However, I concede your point. The core difference is one of accuracy.

deleted 7y ago

[deleted]

sdan 7y ago

It’s all about redundancy. lidars do help though immensely if you integrate it in your ML algorithms currently

vkou 7y ago

> This is a newbie question, but why is the lidar/radar necessary? It's not, in theory. It is, in practice. > > Is it solely because computers aren't yet able to extract as much information from video as humans can from sight? Yes. Currently, you cannot, for any amount of money, buy a camera that has equivalent visual acuity to the human eye.

nneonneo 7y ago
I’m not convinced that the eye is conclusively better than any camera out there. The human brain is incredible at stitching together visual input into a coherent view of the world. The retina has high resolution but covers an incredibly small field of view; rapid eye movements make it possible to fake a larger effective field of view. And I’m not convinced that the retina’s resolution is actually better than that of high-end camera systems. The eye’s performance in the periphery is decidedly worse than that of a wide-angle camera. Dynamic range is a challenge for cameras, but we have high-dynamic-range imaging nowadays (both in software, i.e. exposure combining, and hardware). I don’t think the eye is significantly superior here. Low-light used to be a strength of the human visual system, but I think modern computational imaging systems have caught up (plus, human night vision was really never that good compared with other animals). So in short: I dispute the idea that cameras don’t have the acuity of the human visual system, nowadays. I’d like to know in which aspects you believe human eyes to still be superior, from an optical point of view. Obviously - the human brain and visual cortex is something that computers are nowhere close to.
- vkou 7y ago
  > So in short: I dispute the idea that cameras don’t have the acuity of the human visual system, nowadays. I’d like to know in which aspects you believe human eyes to still be superior, from an optical point of view. Dynamic range. Your eyes can pick up subtle details in a scene with very bright lights, and very dark shadows. No video camera currently exists that can take a good video from inside a city, at night, with a starry night sky. Either the stars, or the lights, or the shadows are going to look like crap. Your eyes can trivially handle such a problem.
7y ago

sumgame 7y ago

It isn't necessary and the best example of this is Tesla's autopilot. It uses no Lidar component and Elon Musk's bet is that self-driving cars can just use just cameras, basic radar and ultrasonic sensors with enough compute resources and the right ML algorithms to perform better than a human. He's been a long time anti lidar proponent because of the costs involved and the aesthetics. He's also betting that the amount of data Tesla receives from its customers, and the neural net they have can achieve autonomous driving with its current hardware stack. “In my view, it’s a crutch that will drive companies to a local maximum that they will find very hard to get out of,” Musk said. He added, “Perhaps I am wrong, and I will look like a fool. But I am quite certain that I am not. "Despite being a fancy and expensive technology, LiDAR provides surprisingly little advantage over a combination of cameras and radar. Radar, for example, is much better in the rain and other limited visibility scenarios, because it is based on radio waves rather than light waves. Radio can penetrate through some objects and bounce back from others, thereby “seeing” the environment along a different dimension." an excerpt from a quora answer on why the Tesla stack could be better than Lidar: https://www.quora.com/Why-dont-Tesla-cars-use-Lidar-like-most-self-driving-cars-in-the-industry?share=1 https://www.quora.com/Why-dont-Tesla-cars-use-Lidar-like-mos... Here's a video of Tesla's autopilot perceives its environment : https://www.youtube.com/watch?v=fKXztwtXaGo https://www.youtube.com/watch?v=fKXztwtXaGo and here's a video of how Waymo perceives its environment: https://www.youtube.com/watch?v=OopTOjnD3qY https://www.youtube.com/watch?v=OopTOjnD3qY The question is if the Lidar adds incremental value or exponential value, and I think it's just incremental by looking at those videos.

karlp 7y ago
A few people died following that bet though
- sumgame 7y ago
  I don't have a Tesla nor am I a fanboy. Just talking about the tech. This is biased information as there are way more Tesla's out there compared to Waymo's. Waymo has had accidents too https://www.wired.com/story/waymo-crash-self-driving-google-arizona/ https://www.wired.com/story/waymo-crash-self-driving-google-...

scarejunba 7y ago

Besides the other reasons it's also that the dynamic range of eyes are way higher than the cameras they can use here. And humans have access to mechanisms that allow them to handle washouts. They're just building in the equivalent of putting in your sunglasses or looking off to the side when the sun is in your eyes or readjusting your seat position so it isn't hitting your eyes.

summerlight 7y ago

Self-driving tech is still at an immature stage while the bar set by public is super high; even human-level self driving might be unacceptable for pervasive adoption. Anyway you can use both Lidar and camera. It's not too late to worry about this cost-performance trade-off after making it actually work.

Traster 7y ago

There are a lot of people here very confidently stating that LIDAR isn't necessary but seemingly forgetting to link to any actual evidence of fully autonomous vehicles that work without it. The issue is that human vision isn't just vision, it's vision linked with a ridiculous general intelligence that we're no where close to reproducing. So humans are great at categorising the objects they see, and reasoning and how they'll behave, which is why we don't generally need LIDAR. So it's just as much about how humans can reason from sight as it is about how good their sight actually is. It's very possible that by the time you've built a computer capable of the same sort of reasoning that humans do, you've actually solved, well, artifical general intelligence and you're no longer designing a car at all - because the AGI took that job ages ago.

nikanj 7y ago
According to your average SV pitch deck, we are 6-10 months away from reproducing general intelligence. Add in a sprinkling of Pascal's Wager, and your funding is almost guaranteed.
- Piskvorrr 7y ago
  Which goes back all the way to the Dartmouth workshop, which hoped to get most of the work done in 6-8 weeks. That was in 1956. Is most of the work done? Well there you go. (Your hint that it's a good way to get paid for such work holds, of course, but the expectation that Problems will get Solved...unlikely)
vbezhenar 7y ago
I'm not sure that human uses that general intelligence when driving and when decisions must be made quickly. I think that brain uses shortcuts and does not really reason about things. And those shortcuts are likely work similar to trained neural networks. That's one reason why novice drivers have problems: they don't have those shortcuts in their brains yet, so they have to reason and that takes valuable time and attention. Though I think that robotic automobiles must perform better than humans, so equipping them with better sensors makes perfect sense, regardless of algorithms.

yogrish 7y ago

Even in broad day light and hith High Res videos and with ML algos, we cannot extract all information accurately that are needed. For example, at what velocity the other vehicle is moving and also at what distance we have an obstacle etc. is very difficult to extract from camera alone and impossible when it is night or extreme weather conditions. Hence we need Radar & Lidars also for ADAS & Self driving applications. Radar: It can accurately measure Distance and Velocity information of objects around ego vehicle and also can track objects. It works well in all weather conditions (Day/night/rain/fog etc). Lidar: Good distance measurement, Rich in data (3D Point cloud) for ML, OK in doing classification (pedestrain, bicyclist etc) even in nights. But expensive sensor.

sreekotay 7y ago

Lots of debate on the topic (clearly :)) But arguably very little progress (none?) in technology, automation, and industrialization has been through dogmatic replication of biological systems. Planes don't fly like birds...