3 ms·

I don't really understand why depth estimation using binocular vision is still a problem. I worked on this a bit a number of years ago and I thought I had scen

I don't really understand why depth estimation using binocular vision is still a problem. I worked on this a bit a number of years ago and I thought I had scene matching working pretty well. The problem is I was trying to make it work without actually having two cameras (ie. on a smartphone where binocular cameras were not available at the time and for the most part still aren't). I was hoping to use the accelerometers and other sensors for dead reckoning the camera's position as the user swept it over a scene, but that turned out to be too hard.

alfanick 6y ago
Isn't your "that turned out to be too hard" one of the answers? People tend to think about machine vision in perfect conditions. This is the problem, me thinks. In practice, you will get a lens flare in most important case, or lose a sensor or get some rain. Instead of designing solutions starting from most hardcore edgecases (which are common with humans), robotics researchers tend to provide MVP that works in best situation. That is far from end-user environmnent. This needs global fixes, if you (I don't care) want robots working in real world.
- tgflynn 6y ago
  But what I said was that the "turned out to be too hard" part had nothing to do with binocular vision. I'm certainly not claiming that I had a fully robust solution but I'm pretty sure lens flares wouldn't be too hard to filter out and there are lots of useful things that need doing in rain-free environments.
- Animats 6y ago
  > Isn't your "that turned out to be too hard" one of the answers? That's a "we need an expert in filter theory" problem. I had four people crash and burn on that problem when we were building a DARPA Grand Challenge vehicle. Combining GPS, accelerometer, gyro, compass, and odometer data to get position is a hard problem. All those sensors are noisy, but in quite different ways. There are off the shelf solutions now, but there were not in 2004. We could not get below 3 degrees of heading noise, and had trouble keeping the sensor map aligned to the real world.
nopeNopeNooope 6y ago
It's not a problem in the sense that it's unsolved. It's a problem in the sense that it takes a rack of machinery that consumes a huge amount of power and doesn't fit well in a mobile, battery powered, form factor. Given that flies do this on uW, obviously there's an approach that is workable, but there's a long path to discovery of what we really need that has to happen. Kind of like how back in the 90s, everyone KNEW you had to use 64bit floats in neural networks to get anything that worked, and now we have carved all that down to the minimum number of bits to make a problem tractable.
rcv 6y ago
Stereo vision isn't exactly "solved," but there are a ton of very good solutions out there both open source and commercial. The "Semi-Global Block Matching" algorithm implementation in the OpenCV library[1] is very good even though the algorithm is over 10 years old at this point. I've played around with the new Intel RealSense[2] units recently as well, which are stereo cameras with onboard processing and a nice IR pattern projector. Pretty cheap, and for the most part they "just work." [1] https://docs.opencv.org/4.4.0/d2/d85/classcv_1_1StereoSGBM.html https://docs.opencv.org/4.4.0/d2/d85/classcv_1_1StereoSGBM.h... [2] https://www.intelrealsense.com/stereo-depth https://www.intelrealsense.com/stereo-depth

alfanick 6y ago

Isn't your "that turned out to be too hard" one of the answers? People tend to think about machine vision in perfect conditions. This is the problem, me thinks. In practice, you will get a lens flare in most important case, or lose a sensor or get some rain. Instead of designing solutions starting from most hardcore edgecases (which are common with humans), robotics researchers tend to provide MVP that works in best situation. That is far from end-user environmnent. This needs global fixes, if you (I don't care) want robots working in real world.

tgflynn 6y ago
But what I said was that the "turned out to be too hard" part had nothing to do with binocular vision. I'm certainly not claiming that I had a fully robust solution but I'm pretty sure lens flares wouldn't be too hard to filter out and there are lots of useful things that need doing in rain-free environments.
Animats 6y ago
> Isn't your "that turned out to be too hard" one of the answers? That's a "we need an expert in filter theory" problem. I had four people crash and burn on that problem when we were building a DARPA Grand Challenge vehicle. Combining GPS, accelerometer, gyro, compass, and odometer data to get position is a hard problem. All those sensors are noisy, but in quite different ways. There are off the shelf solutions now, but there were not in 2004. We could not get below 3 degrees of heading noise, and had trouble keeping the sensor map aligned to the real world.

nopeNopeNooope 6y ago

It's not a problem in the sense that it's unsolved. It's a problem in the sense that it takes a rack of machinery that consumes a huge amount of power and doesn't fit well in a mobile, battery powered, form factor. Given that flies do this on uW, obviously there's an approach that is workable, but there's a long path to discovery of what we really need that has to happen. Kind of like how back in the 90s, everyone KNEW you had to use 64bit floats in neural networks to get anything that worked, and now we have carved all that down to the minimum number of bits to make a problem tractable.

rcv 6y ago

Stereo vision isn't exactly "solved," but there are a ton of very good solutions out there both open source and commercial. The "Semi-Global Block Matching" algorithm implementation in the OpenCV library[1] is very good even though the algorithm is over 10 years old at this point. I've played around with the new Intel RealSense[2] units recently as well, which are stereo cameras with onboard processing and a nice IR pattern projector. Pretty cheap, and for the most part they "just work." [1] https://docs.opencv.org/4.4.0/d2/d85/classcv_1_1StereoSGBM.html https://docs.opencv.org/4.4.0/d2/d85/classcv_1_1StereoSGBM.h... [2] https://www.intelrealsense.com/stereo-depth https://www.intelrealsense.com/stereo-depth