7 ms·
What's a hard case for vision? That's how our eyes work. LiDAR has problems with precipitation too.
by bluthru 10y ago
What's a hard case for vision? That's how our eyes work. LiDAR has problems with precipitation too.
- kchoudhu 10y agoDepth map extraction from vision in real time depends on accurate algorithmic merging of past frames, color gradients and motion vector extraction to come up with a 3D map of what's around the vehicle. Contrast this with LIDAR, which can present a depth map in real time by sending out an array of light pulses and timing how long they take to come back to the IP. Which method is more likely to have implementation errors?
- dolzenko 10y agoSo its not just 2 cameras? Fun, do you have a (entry level) link ?
- kchoudhu 10y agoStereo adds just another layer of data to merge, yeah. I should add a disclaimer: I'm waaaaay out of date. The last time I worked on this stuff was back in 2005 during the DARPA Grand Challenge. And while I'm fairly confident that the basics are still the same (the fact that we're still discussing monocular vs. stereo vs. LIDAR is testament to that...), I'm going to defer to someone more up-to-date to provide more recent research/documentation.
- ilaksh 10y agoThey are using deep learning and also other sensors may contribute to the depth map (assuming they have one).
- Animats 10y agoYeah, me too; I was in the 2005 Grand Challenge. Vision processing has made enormous progress since then. Here's Mobileye's guy explaining what they do and what needs to be done.[1] Here's the NVidia guy.[2] Here's Chris Urmson from Google.[3] All three use their vision system to draw boxes around things that look like obstacles. Then the planner uses those boxes to construct a path. The vision system just gives you a rectangular box in image space; the LIDAR based systems turn point clouds into simple 3D solids. These vision systems do OK on things that look like typical road obstacles - cars and people. Other stuff, not so much. Only Urmson from Google talks about that much. The videos from the vision systems don't seem to be doing much with road edge clutter. They're not marking every post, fireplug, and low-hanging branch. Train a deep learning system on cars, trucks, pedestrians, and bicycles, and your system sees only cars, trucks, pedestrians, and bicycles. These vision systems do a lot less than many people think they do. Watch the videos. [1] https://www.youtube.com/watch?v=GZa9SlMHhQc https://www.youtube.com/watch?v=GZa9SlMHhQc [2] https://www.youtube.com/watch?v=2NGnvGS0AtQ https://www.youtube.com/watch?v=2NGnvGS0AtQ [3] https://www.youtube.com/watch?v=Uj-rK8V-rik https://www.youtube.com/watch?v=Uj-rK8V-rik
- kchoudhu 10y agoAww yeah -- another Challenger! What team were you on?
- Animats 10y agoI ran Team Overbot.[1] We were way overdesigned for off-road and underdesigned for going fast. [1] http://www.overbot.com http://www.overbot.com
- kchoudhu 10y agoSame for us -- speed was not a consideration at all. We figured that if we could keep going and finish, we'd be one of the top 3 teams. As it turned out, our mechanical engineering was great. What got us was software: we failed to free memory for passed obstacles, ran into memory exhaustion issues as a result and crashed out at mile 9. When we fixed the leak and reran the course, the truck finished in just over 7 hours. It was an object lesson that garbage collection won't save you from memory-related issues. In retrospect, it was the also first time I ever encountered a million dollar bug. D'oh.
- TaylorAlexander 10y agoHumans don't use accurate depth maps to drive. Machine learning is great at complex algorithms if done well.
- kchoudhu 10y agoDo the newer systems assume network connectivity to a backend processing facility? Personally, I view each additional layer (network, someone else's data center, machine learning) to be something that can fail and put people in danger. Assurance via local brute force is much more reassuring for me.
- seanmcdirmid 10y agoI can't see a system where processing is done remotely as working very well, nor is it very necessary as the hardware requirements shouldn't be that high. Training could definitely occur in a data center, but that is done offline.
- TaylorAlexander 10y agoNo. The current Tesla Autopilot v2 system has 8 cameras and must be able to react in milliseconds. They do the computation onboard with an Nvidia Drive PX2 computer. https://www.tesla.com/presskit/autopilot#autopilot https://www.tesla.com/presskit/autopilot#autopilot
- onion2k 10y agoHumans don't use accurate depth maps to drive. Humans are very good at estimating distance from a combination parallax visual cues and experience. We don't need to have seen a specific model of car before to judge how far away it is with a high level of accuracy.
- agumonkey 10y agoReminds me how much I hate the average mirror. They force me to lost frontal depth by focusing on sides. I tried driving while looking aside for a minute, it's impressive how much your peripheral vision is meaningless then, even at 5mph you're never sure how far the car before you is.
- sliken 10y agoIn a perfect world lidar is far superior. It's trivial to say "out of the 50M voxels returned, are any from where the car will be in 3 seconds". So lidar wins in the best case, maybe even the average case. However in the real world where the car/sensors gets dirty/wet, and the air is filled with snowflakes, raindrops, or mist it gets much more complicated. Even if statistically better there's value to acting more like a human driver. After all the roads are filled with human controlled cars and any deviation from human norms because of weather(like wind, snow, rain, fog, and blown sand) cause problems.
- piemonkey 10y agoThere are a number of interesting examples in computer vision and psychology literature which document difficult cases for visual perception, see for example slides 14-19 in [0]. Ultimately, vision is a holistic process with edge cases that require knowledge of physics, human psychology, and so-called "common-sense reasoning" in order to resolve. Depending on how one phrases the objective of what a computer vision system should do, the problem can go from a tractable subset of automated reasoning to an intractable general AI task, often with very subtle changes to the problem statement. [0] http://www.cs.toronto.edu/~urtasun/courses/CV/lecture01.pdf http://www.cs.toronto.edu/~urtasun/courses/CV/lecture01.pdf