9 ms·

Naïve question : why is Lidar so important for driverless cars? We human drive cars with only visual and audio information. We don't need lidars. Why couldn'

Naïve question : why is Lidar so important for driverless cars? We human drive cars with only visual and audio information. We don't need lidars. Why couldn't cars do the same?

balazsdavid987 10y ago
The goal is to collect as many information about the environment as possible so you the software can make better decisions than human drivers.
JulianMorrison 10y ago
While I'm no expert, it's obvious right away that humans do a lot of sorts of visual processing that AIs don't on a conceptual, categorizing, and general-knowledge level, and we use these capabilities to interpret visual scenes and draw object boundaries and therefore place objects in modeled 3d space. Example, http://www.optical-illusionist.com/imagefiles/dalmation.jpg http://www.optical-illusionist.com/imagefiles/dalmation.jpg - to see the dog, you really do have to know rather a lot about dogs, sunlight and shadows.
- felipemnoa 10y ago
  >>Example, http://www.optical-illusionist.com/imagefiles/dalmation.jpg http://www.optical-illusionist.com/imagefiles/dalmation.jpg - to see the dog, you really do have to know rather a lot about dogs (Disclaimer: Cynical Plug) This is exactly the sort of AI project that I'm working on: being able to recognize patterns even with lots of noise or incomplete information. I tried a show HN submission [1] but there was absolutely no interest. :( Question: Does anybody know if google is yet at this level? Would google's AI be able to recognize the dalmatian? I think my AI algorithm would be able to recognize the dog but I'm still not at that stage yet. Really need to get more computer power before that. Thanks for the image though, you've given me some really good testing ideas. [1] https://news.ycombinator.com/item?id=14057043 https://news.ycombinator.com/item?id=14057043
  - runako 10y ago
    > Question: Does anybody know if google is yet at this level? Would google's AI be able to recognize the dalmatian? Apparently not, based on trying out the palmation image here: https://cloud.google.com/vision/ https://cloud.google.com/vision/
  - phkahler 10y ago
    If that was not a still image the dog would be far more visible. Once you incorporate motion tracking in 2D many things will suddenly become apparent. Pro-tip: For a moving viewer, anything in a 2D image that doesn't move horizontally or vertically is on a collision course with the camera. This is why deer seem to "come out of nowhere" because they are not moving within the field of view as they run into a car, and could actually remain hidden behind the A-pillar the entire time they are running toward the road right up until you hit em.
pricechild 10y ago
Naïve answer : We humans drive cars despite only visual and audio information? We interpret and often miss things or get fooled by optical illusions. I remember a great story a year or two back about one of Google's cars "weirdly" stopping, before the reporter realised it'd somehow seen through a hedge and noticed a cyclist about to pop into view? I imagine lidar gets you one step closer to the data you're really after.
- hyperbovine 10y ago
  How does lidar see through a hedge?
  - xnzakg 10y ago
    Probably was a radar, not a lidar.
  - mseebach 10y ago
    Possibly infrared sensing? Some fancy IR cameras can see a human body through walls, so I guess it's not inconceivable that a LIDAR (which is already IR, I think?) could penetrate a hedge.
    - swiley 10y ago
      You're thinking of far IR (typical black body radiation at room temperature for most materials) near IR is what has to be filtered out in web cams and what a lot of these LIDAR devices use, it behaves almost exactly like visible light.
  - willvarfar 10y ago
    LIDAR can practically see through vegetation. Some individual beams miss the vegetation and reflect from things they hit on the far side, and the software can detect this. This is used a lot in mapping and research into e.g. forest canopies in the amazon. LIDAR has been extensively deployed on aircraft for this kind of purpose. I've seen high-res LIDAR scans made available by my country's state survey for e.g. archeologists and it has the 'true ground' through the vegetation. Its like the vegetation has been stripped away and has a clinical feel when rendered.
argonaut 10y ago
Why is having a huge cluster of GPUs/CPUs so important for AlphaGo? We humans only using about 100 Watts of power. We don't need megawatts of datacenter power. Visual SLAM is an open research problem, and lidar lets companies sidestep a lot of those problems. I think that's a reasonable bet: falling prices for an existing technology is more predictable than hoping for algorithmic/research advances.
- MichaelBurge 10y ago
  > Why is having a huge cluster of GPUs/CPUs so important for AlphaGo? We humans only using about 100 Watts of power. We don't need megawatts of datacenter power. This argument loses a lot of its bite after AlphaGo beats the top humans. You could argue that you still need lots of energy for training: Even a 100 watt human needs to live for 30/40 years before he's a top Go player. So it's not unreasonable that accelerating the process might use more power. And if you have all that power for training, you might as well use it for inference. You can probably build a bad Go playing machine in much less than 100 watts. There might be a way to extract something more power-efficient from AlphaGo, but it doesn't seem to be important to anyone.
  - BoorishBears 10y ago
    Why can't you apply the same logic in reverse then? We're not able to speed up and compress what takes the 100 Watt machine as much as 21 years to fully develop, into something cars use locally yet, so we sidestep the problem with Lidar. Honestly both seem a little silly though, would an early, poorly trained version of Alpha GO confirm the need for lidar because it doesn't beat top humans?
  - Cybiote 10y ago
    The human brain is doing the playing so we can focus on 20 watts. We can also ignore the on going visual and audio processing, complex motion planning, on line learning, signal filtering and focus on just the Go part (in other words, < 20 watts is dedicated to Go playing). Humans also are unable to learn effectively, due to limits on attention we are not sure of yet, beyond ~4 hours per day. A talented human can reach pro level in about 8 years, enough to beat a neural net only player. The total energy use in joules spent learning is still about an order of magnitude less.
logicallee 10y ago
Hey grondilui it's only 2017 not 2027. "we humans can" doesn't mean computer can. A computer can't even defeat a captcha, many of which are easier than, for example, driving in poor weather under low light conditions. Humans are really bad at a lot of things, such as our reaction time is abysmal, we can be distracted, we can't measure anything perfectly -- but our ability to process visual input (with a bit of delay) into a model of what we're seeing is not one of these areas. Also don't forget that we have stereoscopic vision too and those of us who only have one working eye for some reason are at a disadvantage including when driving. So 3D vision is already necessary for parity with humans. We then do extensive processing not only of still images but parallex effect etc. Computers have trouble even processing a still image. But I find it fascinating that you can already ask "we can do it, why can't computers." you're just a couple of years too soon. :)
kayoone 10y ago
Since Tesla is not using Lidar and they have one of the most advanced self-driving systems, i wonder about this too.
- timr 10y ago
  "Since Tesla is not using Lidar and they have one of the most advanced self-driving systems, i wonder about this too." Who says? The chattering masses on HN? Tesla has certainly deployed a lot of assisted driving systems, but then again, so have a lot of car companies. Lane-keeping and auto-braking are not new, and to date, that's all Tesla has actually shipped. People who know better have serious doubts that you can do full autonomy with only video/radar input under real-world driving conditions (like darkness). That's why most of their competitors are using LIDAR.
  - nojvek 10y ago
    Lidar makes a ton of things easy. Visually extracting 3d world from cameras is a very hard and computationally intensive problem. I really hope velodyne delivers. Quanergy seems to have a nice site but seems vaporware in the sense you can't actually buy it. A $100 light weight Lidar is truly game changing for robots, self driving cars and drones.
    - TillE 10y ago
      > and computationally intensive problem Not an exact analogy, but this reminds me of ancient "software modems", which shaved off a few chips and offloaded processing to the CPU. They were cheap, but had a real impact on your computer's performance. The trouble with image processing is that it seems basically impossible to perfect with the level of certainty you'd need for 100% autonomy, whereas LIDAR gives you very straightforward data to act on. You'd still need the cameras for recognizing traffic lights and such, of course.
    - cr0sh 10y ago
      > A $100 light weight Lidar is truly game changing for robots, self driving cars and drones. This is probably going to end up like the Oculus Rift; it won't be $100.00 - it will probably end up north of $500.00, possibly north of $1000.00. If it were easy, SICK or Hokuyo would have done it already. The fact that neither have can mean many things, of course, but I bet one of the big ones is that it isn't easy to pack a 3D LIDAR into a small package and make it robust and cheap. Both of those companies 2D LIDAR solutions already hit the robust portion; Hokuyo's offerings hit the small package portion (SICK's 2D systems are mostly the size of a coffeemaker - I own a couple), but neither company hits the low price mark. That could also mean that they have a niche market that's willing to pay those prices, but given the interest and want for fast 3D LIDAR for self-driving vehicles and other uses, the fact that they don't have anything out is telling. Now doing it all solid-state? Well - there are companies that have these systems as well (supposedly at least) - called "flash LIDAR"; essentially firing a laser to "flash" the scene, then using a grid-array of CCD-like high-speed elements to gather a 3D delay-time between the flash and reception. From what I've seen, for even the low-resolution modules, they make the former two companies offerings look dirt-cheap in comparison...
Symmetry 10y ago
A human brain has a datacenter's worth of processing power using algorithms tuned by hundreds of millions of years of evolution to be excellent at divining the underlying forms behind visual information. We can't put that on a car (yet). Also, Lidar can avoid some of the optical illusions that will cause humans to crash. E.g. [1] And, hopefully, precise data will let future computerized cars drive more precisely than humans can. [1]http://www.news.com.au/lifestyle/real-life/wtf/optical-illusion-causes-crashes-on-atlanta-freeway/news-story/b7449153b67efb5ddacd07ed3a9941ea http://www.news.com.au/lifestyle/real-life/wtf/optical-illus...
- 50CNT 10y ago
  I was kinda expecting an image showing the optical illusion causing those crashes.
blensor 10y ago
Simple answer: simplicity Our world around us is 3D. A camera projects this information onto a 2D plane and throws away a lot of information in the process. When we move in the world we need to know where obstacles are relative to us. We infer this from all the sensory input we receive and have a pretty good idea how far things are away from us because we have learned how size relations should be. A computer could and can (to some degree) infer this information from regular 2D videos as well, because in the end most algorithms that do obstacle detection/avoidance have to extract positions in 3D to know exactly where those obstacles are. A LIDAR (and other depth sensing technologies) provide a distance for each measurement (sometimes the distance is the only information but some sensors provide intensity, color, etc. as well). So we immediately know the position in space of the measured point and thus can more or less directly extract obstacles in 3D space relative to the robot/car/drone without going the detour of "imagining" 3D information from the projected 2D image. So it is 3D world -> 3D sensor -> 3D points -> 3D obstacles versus 3D world -> 2D sensor(s) -> Extraction of depth -> 3D points -> 3D obstacles
- maxerickson 10y ago
  It's like how we build cars with wheels instead of legs. No reason to stick with our limitations.
  - kbenson 10y ago
    Not so much limitation, as much as optimized for a different problem. Wheels are great for going fast on relatively level terrain, not for traversing any terrain it encounters. Lidar is great for distance representations, but requires a highly consistent and exact timing source. Dual optical inputs allow us to approximate the same data through other methods that don't require nearly as exact timing.
    - maxerickson 10y ago
      Sure, it's the road network that really makes wheels work. But we built that instead of machines with legs.
skdotdan 10y ago
Redundancy?
ceejayoz 10y ago
> We human drive cars with only visual and audio information. And we're kinda shitty at it, killing large numbers of people every year doing it.
- spiderfarmer 10y ago
  I like to think we're kinda amazing at it. The fact that a person can safely drive a car at 80mph while being completely relaxed in a wide range of conditions was really amazing to me as a kid. Accidents happen. Even chairs have their risks.
  - ceejayoz 10y ago
    We kill 30k people a year in the US alone, largely through inattention, speeding, drunk/reckless driving, running lights, etc. All stuff a computer driver isn't likely to do. In 50 years, we'll be shocked humans were ever allowed to do it.
    - spiderfarmer 10y ago
      To be fair, you can already improve that to 10k a year if you learned from European countries like The Netherlands. Better infrastructure, signaling, safety rules and training would help the US a lot. But sure, self driving cars might also be a solution. Who knows. It might certainly be a faster solution.
      - cr0sh 10y ago
        Yeah - we seem to hand out licenses like candy in some or most states here in the US. I'll be the first to say I'm not a great driver. I can and have driven long distances, but I always know I can do better. But all of our training here in the US to pass the test is mainly "ad-hoc". There isn't any kind of "certified training" program that you have to take and pass in order to then get a license. But there probably should be. Honestly, what there really needs to be is such a program, but done over the course of weeks at a facility like what is used at the Bondurant Racing School (https://bondurant.com/ https://bondurant.com/). Having that kind of knowledge and experience to really know your vehicle and its capabilities (both what it can and can't do, and how to handle an emergency) might go a long way toward making better drivers (now that I think of it, I might have to go to that school myself - assuming I can afford it). I think if we did do a better, stricter job at teaching drivers how to drive, and requiring it for a license, it would go a long way to cutting down on accidents. It wouldn't eliminate them, but it would surely reduce them, probably severely. Learning to keep control of emotions, and also handle stressful situations while driving would have to be a part of this as well. It wouldn't be a perfect solution, but it would probably be a damn sight better than what we currently do.
learn_more 10y ago
Stereo vision gives you two slightly different perspectives on a scene. A single point in the 3D scene will be seen in a slightly different position in each 2D image. The offset between the two positions give you the depth information. The more offset, the closer the point is to the observer. The difficult part however is pairing the points between the images. You and I do that easily, because when we see, for example, the top left corner of our refrigerator, we can easily identify that same point in both images, probably because we recognize the objects we are looking at. A computer has more trouble pairing the points. Lidar is a way a cheating, by temporarily shining a dot on the locations so that the points can be found and coordinated easily from both images.
- mturmon 10y ago
  Just to take this a step farther: the distance-measurement error characteristics of lidar and stereo are very different. To get accurate range through stereo, you need two calibrated cameras, rigidly mounted (with respect to each other), and with a high resolution (pixels x pixels in each image of the stereo pair). All this implies very expensive sensors, large camera rigs, and lots of image processing. To get acceptable stopping distances for a vehicle going 60mph using stereo only to detect obstacles requires things like 1.5-meter camera bars, 2048 pixels per image in the stereo pair, and onboard computing to compute stereo correspondence at 30 or 60 fps. It's hardware-intensive! TBH, I forget how the stereo range error scales with range - I think it is linear with range, but may be super-linear. This can be a problem for mapping. I think lidar is superior in this regard, in other words, its error scaling is sub-linear with range. If you're used to using only stereo, the concept of having a lidar for that point-range measurement looks pretty magical. Of course, stereo vision offers some advantages relative to lidar - it's a passive measurement, for example.
  - joshvm 10y ago
    Stereo error is quadratic with range. Bad times. You don't need high resolution, just a wide baseline for accuracy at distance. Resolution will only get you so far because you can't get (and you wouldn't want) sensors with pixels below a micron in size. You usually can't change the focal length much because you have constraints on field of view, therefore the only viable option is more baseline. e_z = e_d * Z^2/(b*f) e_d is the disparity error (i.e. matching error) in metric units; i.e. a few microns, Z is distance, b is baseline, f is focal length It's relatively easy to estimate the accuracy you can achieve on a car because the maximum baseline is fixed to < 2m typically. You can plug in reasonable fields of view, sensors etc. Typically you can assume 0.25 px matching accuracy, assuming the algorithm does sub-pixel interpolation. Example: e_d = 2.2µm, Z=30m, b=1.5m, f=3mm (for a 2000px wide sensor that's 70 degrees FOV): e_z = ~40 cm. That's not too bad - enough to identify something big. At 100 m you'd have a disparity of 20 pixels. If you had a search radius of 256 px you'd have a close-range of 8m. Nowadays the compute problem isn't too bad - you just throw a GPU at it (or an FPGA, but you have memory limitations there). LIDAR has a more or less constant error with range, provided you've got a high enough signal to noise. However this does mean that at short distances, LIDAR is quite poor relatively.
fudged71 10y ago
It's literally an extra dimension of sensing. Yes you can infer distance using parallax (2 regular cameras, like our eyes). But if you can directly SENSE depth using a sensor capable of it (Radar/LIDAR), then you have a lot more certainty, redundancy, simplicity, and can even sense in the dark where cameras aren't able to effectively judge distance.
- ThatGeoGuy 10y ago
  This isn't entirely true. Photogrammetric methods that measure depth from stereo-imagery can and often are more "certain, redundant" and precise when the network is structured correctly. What LiDAR offers here is an easy way to get 3D data quickly without a large computational overhead, and no dependency on camera position. It also means we don't have to solve the correspondence problem, that is, finding the same point on two objects between two images. But LiDAR isn't necessarily enough either. A lot of scanners end up being very noisy, or have a limited range, whereas cameras are good as long as there's line of sight. Using the two technologies together makes a whole lot more sense than just using one or the other. Disclaimer: I am a graduate student who works with LiDAR, photogrammetry, and 3D imaging technologies.
cr0sh 10y ago
First off, having multiple sensors and integration of that data is never a bad thing in robotics; in fact, it is almost a necessary thing. More information can mean better decisions by the system driving the vehicle. More sensors can help fill in things other sensors can't "see". That said, you bring up a good point: Humans do a pretty good job driving vehicles with only a pair of eyeballs (usually), ears, and "feel" (I'm not sure how to put this last one, but a good driver tries to stay and feel "as one" with the vehicle, getting information about the road, various forces internal and external about the car, etc - in order to drive it properly and in control while making various maneuvers). So why couldn't we do the same with a self-driving vehicle? Well - the truth is - we can: https://devblogs.nvidia.com/parallelforall/deep-learning-self-driving-cars/ https://devblogs.nvidia.com/parallelforall/deep-learning-sel... https://images.nvidia.com/content/tegra/automotive/images/2016/solutions/pdf/end-to-end-dl-using-px.pdf https://images.nvidia.com/content/tegra/automotive/images/20... I mention that paper so often in these kinds of discussions, that I am sure people are getting sick of it. Note that in that paper, only a single camera is used - like driving with one eye closed (or blind in one eye), yet it still was able to do so properly. Part of Nvidia's purpose here is to advance self-driving vehicles, but they also are trying to sell self-driving vehicle manufacturers on their tech as well: http://www.nvidia.com/object/drive-px.html http://www.nvidia.com/object/drive-px.html If the need for expensive and exotic LIDAR systems can be overcome, and simpler systems like cameras and radar can instead be used, it will both be a cheaper way to manufacture these cars, as well as being more reliable (as the article noted, it's one thing to make a LIDAR - making a reliable LIDAR for automotive use over 100s of thousands of miles/kilometers is a different ball-o-wax - but manufacturers already have experience with making radar and cameras for vehicles robust). For another take on using cameras for autonomous navigation - there's this project using high-speed winged drones (and completely open source - including the drone design files): http://spectrum.ieee.org/automaton/robotics/drones/mit-drone-avoids-obstacles http://spectrum.ieee.org/automaton/robotics/drones/mit-drone... https://github.com/andybarry/flight https://github.com/andybarry/flight That one only uses a couple of small cameras for stereo vision. These two projects (and others if you care to search) both show that such a system is possible; we probably don't need LIDAR for the majority of driving scenarios. Radar can probably cover other areas, and you mention the concept of humans using hearing - but honestly I am not aware of any self-driving vehicle research on that potential kind of sensing, but if none exists, it seems like it might be a fascinating and promising area of study for someone! Audio sensors would also be another one of those "simple and robust" sensors for automobile usage that manufacturers would like. That said: If they can make a simple but robust LIDAR system, with no moving parts (such designs do exist, like flash LIDAR, for instance), and a low-cost (that part is key), having that on a vehicle certainly can't hurt. There are areas (particularly at night, or in inclement weather, or in other low-visibility situations) where a camera alone will have a similar trouble as a human would (though I wonder if a camera operating at other wavelengths (FLIR, Near-IR, and Near-UV for instance) could help?) In those situations, having a LIDAR might be the key to a safer driving experience for a self-driving vehicle, whereas a human without that ability might make a wrong (and potentially fatal) decision.
devy 10y ago
Self-driving cars need to be safer and more reliable for the regulators and mass to adopt. Humans are pretty bad drivers with only visual + audio information when one of both are impaired by natural conditions (weather, night time etc.) or artificial conditions (DUIs etc.). By introducing more dimensions of sensing (LiDAR, Sonar etc.) as some others put it, it should theoretically help judgement during driving and thus improve the safety aspect.
danblick 10y ago
In the curriculum for Udacity's self-driving car nanodegree program, your suggestion was made explicitly. ~"(re: why are we focusing on vision?) Humans drive with just our eyes (even one eye would work), perhaps in the future cars will need only camera data instead." (Perhaps many cameras.)
joshvm 10y ago
One big benefit of LIDAR is that it's an active ranging technique - that is, it illuminates its target. That means it works at night and whereas a human would need 360º headlights, a LIDAR doesn't. The second benefit is that LIDAR is robust and quite accurate. If you measure a distance, you know it's probably right and with what confidence. With vision based methods, you have trouble reconstructing featureless surfaces. Finally as others have pointed out, LIDAR gives you 3D information out of the box. You don't need to do a bunch of compute-intensive image matching to get range. It's much more likely that a stereo vision based system will give you false positives due to an incorrect feature match. The main downside compared to cameras is that LIDAR data is much sparser. You can get 1MPt/sec LIDAR, but that's over an entire hemisphere normally so in one camera's field of view you might only get 20k measurements compared to millions of pixels. In the end you need both (plus RADAR, ultrasound and everything else). Cameras will always be required for tasks like lane marking detection, reading road signs and visual identification of obstacles like pedestrians.

Naïve question : why is Lidar so important for driverless cars? We human drive cars with only visual and audio information. We don't need lidars. Why couldn't cars do the same?

balazsdavid987 10y ago

The goal is to collect as many information about the environment as possible so you the software can make better decisions than human drivers.

JulianMorrison 10y ago

While I'm no expert, it's obvious right away that humans do a lot of sorts of visual processing that AIs don't on a conceptual, categorizing, and general-knowledge level, and we use these capabilities to interpret visual scenes and draw object boundaries and therefore place objects in modeled 3d space. Example, http://www.optical-illusionist.com/imagefiles/dalmation.jpg http://www.optical-illusionist.com/imagefiles/dalmation.jpg - to see the dog, you really do have to know rather a lot about dogs, sunlight and shadows.

felipemnoa 10y ago
>>Example, http://www.optical-illusionist.com/imagefiles/dalmation.jpg http://www.optical-illusionist.com/imagefiles/dalmation.jpg - to see the dog, you really do have to know rather a lot about dogs (Disclaimer: Cynical Plug) This is exactly the sort of AI project that I'm working on: being able to recognize patterns even with lots of noise or incomplete information. I tried a show HN submission [1] but there was absolutely no interest. :( Question: Does anybody know if google is yet at this level? Would google's AI be able to recognize the dalmatian? I think my AI algorithm would be able to recognize the dog but I'm still not at that stage yet. Really need to get more computer power before that. Thanks for the image though, you've given me some really good testing ideas. [1] https://news.ycombinator.com/item?id=14057043 https://news.ycombinator.com/item?id=14057043
- runako 10y ago
  > Question: Does anybody know if google is yet at this level? Would google's AI be able to recognize the dalmatian? Apparently not, based on trying out the palmation image here: https://cloud.google.com/vision/ https://cloud.google.com/vision/
- phkahler 10y ago
  If that was not a still image the dog would be far more visible. Once you incorporate motion tracking in 2D many things will suddenly become apparent. Pro-tip: For a moving viewer, anything in a 2D image that doesn't move horizontally or vertically is on a collision course with the camera. This is why deer seem to "come out of nowhere" because they are not moving within the field of view as they run into a car, and could actually remain hidden behind the A-pillar the entire time they are running toward the road right up until you hit em.

pricechild 10y ago

Naïve answer : We humans drive cars despite only visual and audio information? We interpret and often miss things or get fooled by optical illusions. I remember a great story a year or two back about one of Google's cars "weirdly" stopping, before the reporter realised it'd somehow seen through a hedge and noticed a cyclist about to pop into view? I imagine lidar gets you one step closer to the data you're really after.

hyperbovine 10y ago
How does lidar see through a hedge?
- xnzakg 10y ago
  Probably was a radar, not a lidar.
- mseebach 10y ago
  Possibly infrared sensing? Some fancy IR cameras can see a human body through walls, so I guess it's not inconceivable that a LIDAR (which is already IR, I think?) could penetrate a hedge.
  - swiley 10y ago
    You're thinking of far IR (typical black body radiation at room temperature for most materials) near IR is what has to be filtered out in web cams and what a lot of these LIDAR devices use, it behaves almost exactly like visible light.
- willvarfar 10y ago
  LIDAR can practically see through vegetation. Some individual beams miss the vegetation and reflect from things they hit on the far side, and the software can detect this. This is used a lot in mapping and research into e.g. forest canopies in the amazon. LIDAR has been extensively deployed on aircraft for this kind of purpose. I've seen high-res LIDAR scans made available by my country's state survey for e.g. archeologists and it has the 'true ground' through the vegetation. Its like the vegetation has been stripped away and has a clinical feel when rendered.

argonaut 10y ago

Why is having a huge cluster of GPUs/CPUs so important for AlphaGo? We humans only using about 100 Watts of power. We don't need megawatts of datacenter power. Visual SLAM is an open research problem, and lidar lets companies sidestep a lot of those problems. I think that's a reasonable bet: falling prices for an existing technology is more predictable than hoping for algorithmic/research advances.

MichaelBurge 10y ago
> Why is having a huge cluster of GPUs/CPUs so important for AlphaGo? We humans only using about 100 Watts of power. We don't need megawatts of datacenter power. This argument loses a lot of its bite after AlphaGo beats the top humans. You could argue that you still need lots of energy for training: Even a 100 watt human needs to live for 30/40 years before he's a top Go player. So it's not unreasonable that accelerating the process might use more power. And if you have all that power for training, you might as well use it for inference. You can probably build a bad Go playing machine in much less than 100 watts. There might be a way to extract something more power-efficient from AlphaGo, but it doesn't seem to be important to anyone.
- BoorishBears 10y ago
  Why can't you apply the same logic in reverse then? We're not able to speed up and compress what takes the 100 Watt machine as much as 21 years to fully develop, into something cars use locally yet, so we sidestep the problem with Lidar. Honestly both seem a little silly though, would an early, poorly trained version of Alpha GO confirm the need for lidar because it doesn't beat top humans?
- Cybiote 10y ago
  The human brain is doing the playing so we can focus on 20 watts. We can also ignore the on going visual and audio processing, complex motion planning, on line learning, signal filtering and focus on just the Go part (in other words, < 20 watts is dedicated to Go playing). Humans also are unable to learn effectively, due to limits on attention we are not sure of yet, beyond ~4 hours per day. A talented human can reach pro level in about 8 years, enough to beat a neural net only player. The total energy use in joules spent learning is still about an order of magnitude less.

logicallee 10y ago

Hey grondilui it's only 2017 not 2027. "we humans can" doesn't mean computer can. A computer can't even defeat a captcha, many of which are easier than, for example, driving in poor weather under low light conditions. Humans are really bad at a lot of things, such as our reaction time is abysmal, we can be distracted, we can't measure anything perfectly -- but our ability to process visual input (with a bit of delay) into a model of what we're seeing is not one of these areas. Also don't forget that we have stereoscopic vision too and those of us who only have one working eye for some reason are at a disadvantage including when driving. So 3D vision is already necessary for parity with humans. We then do extensive processing not only of still images but parallex effect etc. Computers have trouble even processing a still image. But I find it fascinating that you can already ask "we can do it, why can't computers." you're just a couple of years too soon. :)

kayoone 10y ago

Since Tesla is not using Lidar and they have one of the most advanced self-driving systems, i wonder about this too.

timr 10y ago
"Since Tesla is not using Lidar and they have one of the most advanced self-driving systems, i wonder about this too." Who says? The chattering masses on HN? Tesla has certainly deployed a lot of assisted driving systems, but then again, so have a lot of car companies. Lane-keeping and auto-braking are not new, and to date, that's all Tesla has actually shipped. People who know better have serious doubts that you can do full autonomy with only video/radar input under real-world driving conditions (like darkness). That's why most of their competitors are using LIDAR.
- nojvek 10y ago
  Lidar makes a ton of things easy. Visually extracting 3d world from cameras is a very hard and computationally intensive problem. I really hope velodyne delivers. Quanergy seems to have a nice site but seems vaporware in the sense you can't actually buy it. A $100 light weight Lidar is truly game changing for robots, self driving cars and drones.
  - TillE 10y ago
    > and computationally intensive problem Not an exact analogy, but this reminds me of ancient "software modems", which shaved off a few chips and offloaded processing to the CPU. They were cheap, but had a real impact on your computer's performance. The trouble with image processing is that it seems basically impossible to perfect with the level of certainty you'd need for 100% autonomy, whereas LIDAR gives you very straightforward data to act on. You'd still need the cameras for recognizing traffic lights and such, of course.
  - cr0sh 10y ago
    > A $100 light weight Lidar is truly game changing for robots, self driving cars and drones. This is probably going to end up like the Oculus Rift; it won't be $100.00 - it will probably end up north of $500.00, possibly north of $1000.00. If it were easy, SICK or Hokuyo would have done it already. The fact that neither have can mean many things, of course, but I bet one of the big ones is that it isn't easy to pack a 3D LIDAR into a small package and make it robust and cheap. Both of those companies 2D LIDAR solutions already hit the robust portion; Hokuyo's offerings hit the small package portion (SICK's 2D systems are mostly the size of a coffeemaker - I own a couple), but neither company hits the low price mark. That could also mean that they have a niche market that's willing to pay those prices, but given the interest and want for fast 3D LIDAR for self-driving vehicles and other uses, the fact that they don't have anything out is telling. Now doing it all solid-state? Well - there are companies that have these systems as well (supposedly at least) - called "flash LIDAR"; essentially firing a laser to "flash" the scene, then using a grid-array of CCD-like high-speed elements to gather a 3D delay-time between the flash and reception. From what I've seen, for even the low-resolution modules, they make the former two companies offerings look dirt-cheap in comparison...

Symmetry 10y ago

A human brain has a datacenter's worth of processing power using algorithms tuned by hundreds of millions of years of evolution to be excellent at divining the underlying forms behind visual information. We can't put that on a car (yet). Also, Lidar can avoid some of the optical illusions that will cause humans to crash. E.g. [1] And, hopefully, precise data will let future computerized cars drive more precisely than humans can. [1]http://www.news.com.au/lifestyle/real-life/wtf/optical-illusion-causes-crashes-on-atlanta-freeway/news-story/b7449153b67efb5ddacd07ed3a9941ea http://www.news.com.au/lifestyle/real-life/wtf/optical-illus...

50CNT 10y ago
I was kinda expecting an image showing the optical illusion causing those crashes.

blensor 10y ago

Simple answer: simplicity Our world around us is 3D. A camera projects this information onto a 2D plane and throws away a lot of information in the process. When we move in the world we need to know where obstacles are relative to us. We infer this from all the sensory input we receive and have a pretty good idea how far things are away from us because we have learned how size relations should be. A computer could and can (to some degree) infer this information from regular 2D videos as well, because in the end most algorithms that do obstacle detection/avoidance have to extract positions in 3D to know exactly where those obstacles are. A LIDAR (and other depth sensing technologies) provide a distance for each measurement (sometimes the distance is the only information but some sensors provide intensity, color, etc. as well). So we immediately know the position in space of the measured point and thus can more or less directly extract obstacles in 3D space relative to the robot/car/drone without going the detour of "imagining" 3D information from the projected 2D image. So it is 3D world -> 3D sensor -> 3D points -> 3D obstacles versus 3D world -> 2D sensor(s) -> Extraction of depth -> 3D points -> 3D obstacles

maxerickson 10y ago
It's like how we build cars with wheels instead of legs. No reason to stick with our limitations.
- kbenson 10y ago
  Not so much limitation, as much as optimized for a different problem. Wheels are great for going fast on relatively level terrain, not for traversing any terrain it encounters. Lidar is great for distance representations, but requires a highly consistent and exact timing source. Dual optical inputs allow us to approximate the same data through other methods that don't require nearly as exact timing.
  - maxerickson 10y ago
    Sure, it's the road network that really makes wheels work. But we built that instead of machines with legs.

skdotdan 10y ago

Redundancy?

ceejayoz 10y ago

> We human drive cars with only visual and audio information. And we're kinda shitty at it, killing large numbers of people every year doing it.

spiderfarmer 10y ago
I like to think we're kinda amazing at it. The fact that a person can safely drive a car at 80mph while being completely relaxed in a wide range of conditions was really amazing to me as a kid. Accidents happen. Even chairs have their risks.
- ceejayoz 10y ago
  We kill 30k people a year in the US alone, largely through inattention, speeding, drunk/reckless driving, running lights, etc. All stuff a computer driver isn't likely to do. In 50 years, we'll be shocked humans were ever allowed to do it.
  - spiderfarmer 10y ago
    To be fair, you can already improve that to 10k a year if you learned from European countries like The Netherlands. Better infrastructure, signaling, safety rules and training would help the US a lot. But sure, self driving cars might also be a solution. Who knows. It might certainly be a faster solution.
    - cr0sh 10y ago
      Yeah - we seem to hand out licenses like candy in some or most states here in the US. I'll be the first to say I'm not a great driver. I can and have driven long distances, but I always know I can do better. But all of our training here in the US to pass the test is mainly "ad-hoc". There isn't any kind of "certified training" program that you have to take and pass in order to then get a license. But there probably should be. Honestly, what there really needs to be is such a program, but done over the course of weeks at a facility like what is used at the Bondurant Racing School (https://bondurant.com/ https://bondurant.com/). Having that kind of knowledge and experience to really know your vehicle and its capabilities (both what it can and can't do, and how to handle an emergency) might go a long way toward making better drivers (now that I think of it, I might have to go to that school myself - assuming I can afford it). I think if we did do a better, stricter job at teaching drivers how to drive, and requiring it for a license, it would go a long way to cutting down on accidents. It wouldn't eliminate them, but it would surely reduce them, probably severely. Learning to keep control of emotions, and also handle stressful situations while driving would have to be a part of this as well. It wouldn't be a perfect solution, but it would probably be a damn sight better than what we currently do.

learn_more 10y ago

Stereo vision gives you two slightly different perspectives on a scene. A single point in the 3D scene will be seen in a slightly different position in each 2D image. The offset between the two positions give you the depth information. The more offset, the closer the point is to the observer. The difficult part however is pairing the points between the images. You and I do that easily, because when we see, for example, the top left corner of our refrigerator, we can easily identify that same point in both images, probably because we recognize the objects we are looking at. A computer has more trouble pairing the points. Lidar is a way a cheating, by temporarily shining a dot on the locations so that the points can be found and coordinated easily from both images.

mturmon 10y ago
Just to take this a step farther: the distance-measurement error characteristics of lidar and stereo are very different. To get accurate range through stereo, you need two calibrated cameras, rigidly mounted (with respect to each other), and with a high resolution (pixels x pixels in each image of the stereo pair). All this implies very expensive sensors, large camera rigs, and lots of image processing. To get acceptable stopping distances for a vehicle going 60mph using stereo only to detect obstacles requires things like 1.5-meter camera bars, 2048 pixels per image in the stereo pair, and onboard computing to compute stereo correspondence at 30 or 60 fps. It's hardware-intensive! TBH, I forget how the stereo range error scales with range - I think it is linear with range, but may be super-linear. This can be a problem for mapping. I think lidar is superior in this regard, in other words, its error scaling is sub-linear with range. If you're used to using only stereo, the concept of having a lidar for that point-range measurement looks pretty magical. Of course, stereo vision offers some advantages relative to lidar - it's a passive measurement, for example.
- joshvm 10y ago
  Stereo error is quadratic with range. Bad times. You don't need high resolution, just a wide baseline for accuracy at distance. Resolution will only get you so far because you can't get (and you wouldn't want) sensors with pixels below a micron in size. You usually can't change the focal length much because you have constraints on field of view, therefore the only viable option is more baseline. e_z = e_d * Z^2/(b*f) e_d is the disparity error (i.e. matching error) in metric units; i.e. a few microns, Z is distance, b is baseline, f is focal length It's relatively easy to estimate the accuracy you can achieve on a car because the maximum baseline is fixed to < 2m typically. You can plug in reasonable fields of view, sensors etc. Typically you can assume 0.25 px matching accuracy, assuming the algorithm does sub-pixel interpolation. Example: e_d = 2.2µm, Z=30m, b=1.5m, f=3mm (for a 2000px wide sensor that's 70 degrees FOV): e_z = ~40 cm. That's not too bad - enough to identify something big. At 100 m you'd have a disparity of 20 pixels. If you had a search radius of 256 px you'd have a close-range of 8m. Nowadays the compute problem isn't too bad - you just throw a GPU at it (or an FPGA, but you have memory limitations there). LIDAR has a more or less constant error with range, provided you've got a high enough signal to noise. However this does mean that at short distances, LIDAR is quite poor relatively.

fudged71 10y ago

It's literally an extra dimension of sensing. Yes you can infer distance using parallax (2 regular cameras, like our eyes). But if you can directly SENSE depth using a sensor capable of it (Radar/LIDAR), then you have a lot more certainty, redundancy, simplicity, and can even sense in the dark where cameras aren't able to effectively judge distance.

ThatGeoGuy 10y ago
This isn't entirely true. Photogrammetric methods that measure depth from stereo-imagery can and often are more "certain, redundant" and precise when the network is structured correctly. What LiDAR offers here is an easy way to get 3D data quickly without a large computational overhead, and no dependency on camera position. It also means we don't have to solve the correspondence problem, that is, finding the same point on two objects between two images. But LiDAR isn't necessarily enough either. A lot of scanners end up being very noisy, or have a limited range, whereas cameras are good as long as there's line of sight. Using the two technologies together makes a whole lot more sense than just using one or the other. Disclaimer: I am a graduate student who works with LiDAR, photogrammetry, and 3D imaging technologies.

cr0sh 10y ago

First off, having multiple sensors and integration of that data is never a bad thing in robotics; in fact, it is almost a necessary thing. More information can mean better decisions by the system driving the vehicle. More sensors can help fill in things other sensors can't "see". That said, you bring up a good point: Humans do a pretty good job driving vehicles with only a pair of eyeballs (usually), ears, and "feel" (I'm not sure how to put this last one, but a good driver tries to stay and feel "as one" with the vehicle, getting information about the road, various forces internal and external about the car, etc - in order to drive it properly and in control while making various maneuvers). So why couldn't we do the same with a self-driving vehicle? Well - the truth is - we can: https://devblogs.nvidia.com/parallelforall/deep-learning-self-driving-cars/ https://devblogs.nvidia.com/parallelforall/deep-learning-sel... https://images.nvidia.com/content/tegra/automotive/images/2016/solutions/pdf/end-to-end-dl-using-px.pdf https://images.nvidia.com/content/tegra/automotive/images/20... I mention that paper so often in these kinds of discussions, that I am sure people are getting sick of it. Note that in that paper, only a single camera is used - like driving with one eye closed (or blind in one eye), yet it still was able to do so properly. Part of Nvidia's purpose here is to advance self-driving vehicles, but they also are trying to sell self-driving vehicle manufacturers on their tech as well: http://www.nvidia.com/object/drive-px.html http://www.nvidia.com/object/drive-px.html If the need for expensive and exotic LIDAR systems can be overcome, and simpler systems like cameras and radar can instead be used, it will both be a cheaper way to manufacture these cars, as well as being more reliable (as the article noted, it's one thing to make a LIDAR - making a reliable LIDAR for automotive use over 100s of thousands of miles/kilometers is a different ball-o-wax - but manufacturers already have experience with making radar and cameras for vehicles robust). For another take on using cameras for autonomous navigation - there's this project using high-speed winged drones (and completely open source - including the drone design files): http://spectrum.ieee.org/automaton/robotics/drones/mit-drone-avoids-obstacles http://spectrum.ieee.org/automaton/robotics/drones/mit-drone... https://github.com/andybarry/flight https://github.com/andybarry/flight That one only uses a couple of small cameras for stereo vision. These two projects (and others if you care to search) both show that such a system is possible; we probably don't need LIDAR for the majority of driving scenarios. Radar can probably cover other areas, and you mention the concept of humans using hearing - but honestly I am not aware of any self-driving vehicle research on that potential kind of sensing, but if none exists, it seems like it might be a fascinating and promising area of study for someone! Audio sensors would also be another one of those "simple and robust" sensors for automobile usage that manufacturers would like. That said: If they can make a simple but robust LIDAR system, with no moving parts (such designs do exist, like flash LIDAR, for instance), and a low-cost (that part is key), having that on a vehicle certainly can't hurt. There are areas (particularly at night, or in inclement weather, or in other low-visibility situations) where a camera alone will have a similar trouble as a human would (though I wonder if a camera operating at other wavelengths (FLIR, Near-IR, and Near-UV for instance) could help?) In those situations, having a LIDAR might be the key to a safer driving experience for a self-driving vehicle, whereas a human without that ability might make a wrong (and potentially fatal) decision.

devy 10y ago

Self-driving cars need to be safer and more reliable for the regulators and mass to adopt. Humans are pretty bad drivers with only visual + audio information when one of both are impaired by natural conditions (weather, night time etc.) or artificial conditions (DUIs etc.). By introducing more dimensions of sensing (LiDAR, Sonar etc.) as some others put it, it should theoretically help judgement during driving and thus improve the safety aspect.

danblick 10y ago

In the curriculum for Udacity's self-driving car nanodegree program, your suggestion was made explicitly. ~"(re: why are we focusing on vision?) Humans drive with just our eyes (even one eye would work), perhaps in the future cars will need only camera data instead." (Perhaps many cameras.)

joshvm 10y ago

One big benefit of LIDAR is that it's an active ranging technique - that is, it illuminates its target. That means it works at night and whereas a human would need 360º headlights, a LIDAR doesn't. The second benefit is that LIDAR is robust and quite accurate. If you measure a distance, you know it's probably right and with what confidence. With vision based methods, you have trouble reconstructing featureless surfaces. Finally as others have pointed out, LIDAR gives you 3D information out of the box. You don't need to do a bunch of compute-intensive image matching to get range. It's much more likely that a stereo vision based system will give you false positives due to an incorrect feature match. The main downside compared to cameras is that LIDAR data is much sparser. You can get 1MPt/sec LIDAR, but that's over an entire hemisphere normally so in one camera's field of view you might only get 20k measurements compared to millions of pixels. In the end you need both (plus RADAR, ultrasound and everything else). Cameras will always be required for tasks like lane marking detection, reading road signs and visual identification of obstacles like pedestrians.