3 ms·

So its not just 2 cameras? Fun, do you have a (entry level) link ?

kchoudhu 10y ago
Stereo adds just another layer of data to merge, yeah. I should add a disclaimer: I'm waaaaay out of date. The last time I worked on this stuff was back in 2005 during the DARPA Grand Challenge. And while I'm fairly confident that the basics are still the same (the fact that we're still discussing monocular vs. stereo vs. LIDAR is testament to that...), I'm going to defer to someone more up-to-date to provide more recent research/documentation.
- ilaksh 10y ago
  They are using deep learning and also other sensors may contribute to the depth map (assuming they have one).
- Animats 10y ago
  Yeah, me too; I was in the 2005 Grand Challenge. Vision processing has made enormous progress since then. Here's Mobileye's guy explaining what they do and what needs to be done.[1] Here's the NVidia guy.[2] Here's Chris Urmson from Google.[3] All three use their vision system to draw boxes around things that look like obstacles. Then the planner uses those boxes to construct a path. The vision system just gives you a rectangular box in image space; the LIDAR based systems turn point clouds into simple 3D solids. These vision systems do OK on things that look like typical road obstacles - cars and people. Other stuff, not so much. Only Urmson from Google talks about that much. The videos from the vision systems don't seem to be doing much with road edge clutter. They're not marking every post, fireplug, and low-hanging branch. Train a deep learning system on cars, trucks, pedestrians, and bicycles, and your system sees only cars, trucks, pedestrians, and bicycles. These vision systems do a lot less than many people think they do. Watch the videos. [1] https://www.youtube.com/watch?v=GZa9SlMHhQc https://www.youtube.com/watch?v=GZa9SlMHhQc [2] https://www.youtube.com/watch?v=2NGnvGS0AtQ https://www.youtube.com/watch?v=2NGnvGS0AtQ [3] https://www.youtube.com/watch?v=Uj-rK8V-rik https://www.youtube.com/watch?v=Uj-rK8V-rik
  - kchoudhu 10y ago
    Aww yeah -- another Challenger! What team were you on?
    - Animats 10y ago
      I ran Team Overbot.[1] We were way overdesigned for off-road and underdesigned for going fast. [1] http://www.overbot.com http://www.overbot.com
      - kchoudhu 10y ago
        Same for us -- speed was not a consideration at all. We figured that if we could keep going and finish, we'd be one of the top 3 teams. As it turned out, our mechanical engineering was great. What got us was software: we failed to free memory for passed obstacles, ran into memory exhaustion issues as a result and crashed out at mile 9. When we fixed the leak and reran the course, the truck finished in just over 7 hours. It was an object lesson that garbage collection won't save you from memory-related issues. In retrospect, it was the also first time I ever encountered a million dollar bug. D'oh.
        dzhiurgis 10y ago
        What was the second time?
        kchoudhu 10y ago
        I ran across quite a few of them while working tech at an investment bank. Never anything that resulted in direct losses, but implied missed revenue -- definitely.

Stereo adds just another layer of data to merge, yeah. I should add a disclaimer: I'm waaaaay out of date. The last time I worked on this stuff was back in 2005 during the DARPA Grand Challenge. And while I'm fairly confident that the basics are still the same (the fact that we're still discussing monocular vs. stereo vs. LIDAR is testament to that...), I'm going to defer to someone more up-to-date to provide more recent research/documentation.

ilaksh 10y ago
They are using deep learning and also other sensors may contribute to the depth map (assuming they have one).
Animats 10y ago
Yeah, me too; I was in the 2005 Grand Challenge. Vision processing has made enormous progress since then. Here's Mobileye's guy explaining what they do and what needs to be done.[1] Here's the NVidia guy.[2] Here's Chris Urmson from Google.[3] All three use their vision system to draw boxes around things that look like obstacles. Then the planner uses those boxes to construct a path. The vision system just gives you a rectangular box in image space; the LIDAR based systems turn point clouds into simple 3D solids. These vision systems do OK on things that look like typical road obstacles - cars and people. Other stuff, not so much. Only Urmson from Google talks about that much. The videos from the vision systems don't seem to be doing much with road edge clutter. They're not marking every post, fireplug, and low-hanging branch. Train a deep learning system on cars, trucks, pedestrians, and bicycles, and your system sees only cars, trucks, pedestrians, and bicycles. These vision systems do a lot less than many people think they do. Watch the videos. [1] https://www.youtube.com/watch?v=GZa9SlMHhQc https://www.youtube.com/watch?v=GZa9SlMHhQc [2] https://www.youtube.com/watch?v=2NGnvGS0AtQ https://www.youtube.com/watch?v=2NGnvGS0AtQ [3] https://www.youtube.com/watch?v=Uj-rK8V-rik https://www.youtube.com/watch?v=Uj-rK8V-rik
- kchoudhu 10y ago
  Aww yeah -- another Challenger! What team were you on?
  - Animats 10y ago
    I ran Team Overbot.[1] We were way overdesigned for off-road and underdesigned for going fast. [1] http://www.overbot.com http://www.overbot.com
    - kchoudhu 10y ago
      Same for us -- speed was not a consideration at all. We figured that if we could keep going and finish, we'd be one of the top 3 teams. As it turned out, our mechanical engineering was great. What got us was software: we failed to free memory for passed obstacles, ran into memory exhaustion issues as a result and crashed out at mile 9. When we fixed the leak and reran the course, the truck finished in just over 7 hours. It was an object lesson that garbage collection won't save you from memory-related issues. In retrospect, it was the also first time I ever encountered a million dollar bug. D'oh.
      - dzhiurgis 10y ago
        What was the second time?
        kchoudhu 10y ago
        I ran across quite a few of them while working tech at an investment bank. Never anything that resulted in direct losses, but implied missed revenue -- definitely.