4 ms·

Another bit of code in this space is libmv: https://github.com/libmv/libmv https://github.com/libmv/libmv. libmv's codebase seems to be forked, with an earlier

by femto 13y ago

Another bit of code in this space is libmv: https://github.com/libmv/libmv https://github.com/libmv/libmv. libmv's codebase seems to be forked, with an earlier version at goggle code: http://code.google.com/p/libmv/ http://code.google.com/p/libmv/ which also contains an interesting summary of other libraries in the 3D reconstruction space. Blender also has its own fork, which it uses for matchmoving, which is the integration of animated objects into a real world scene. In turn, libmv seems be be influenced by the work of Marc Pollefeys? The tutorial is a readable summary of how to go from a collection of 2D images to a 3D model. http://www.cs.unc.edu/~marc/research.html http://www.cs.unc.edu/~marc/research.html http://www.cs.unc.edu/~marc/tutorial/ http://www.cs.unc.edu/~marc/tutorial/ http://www.cs.unc.edu/~marc/tutorial.pdf http://www.cs.unc.edu/~marc/tutorial.pdf Question: Can a knowledgeable person here suggest which codebase is the best to start experimenting with, to build an application that converts a 2D photo sequence into a dimensionally accurate 3D model?

apu 13y ago
Although I don't have first hand experience with the code, a former labmate of mine (I'm at the Univ. of Washington, and he's now at Google) is one of the leading experts on this area of research, and his VisualSFM [1] tool is, I think, the best and easiest-to-use available online. Briefly, there are three main steps required to go from images to a 3d viewer like PhotoSynth: 1. Figure out where each image was shot from (the "camera pose") and get a sparse set of 3d points from the scene. These two are estimated simultaneously using bundle adjustment [2]. 2. Go from a sparse set of 3d points to a dense 3d model. This is done using a technique called Multiple View Stereo (MVS), of which the leading (open) implementations are PMVS/CMVS [3,4]. 3. Build an image-based rendering system that intelligently blends between the 3d models and images to minimize artifacts. The VisualSFM software will do steps 1 and 2. Step 3 is still quite a challenging problem, but depending on what you're doing, you could use standard 3d modeling environments to look at your data. [1] http://ccwu.me/vsfm/ http://ccwu.me/vsfm/ [2] http://en.wikipedia.org/wiki/Bundle_adjustment http://en.wikipedia.org/wiki/Bundle_adjustment [3] http://www.di.ens.fr/pmvs/ http://www.di.ens.fr/pmvs/ [4] http://www.di.ens.fr/cmvs/ http://www.di.ens.fr/cmvs/
amelim 13y ago
Sure! My lab focuses on SLAM and 3D reconstruction, especially for robotic applications. We've developed a BSD-licensed C++ (w/ MATLAB wrapper) library with specific applications towards 3D reconstruction problems such as SLAM and structure from motion. It's called GTSAM [1] We actively maintain and release new features as they are published. While we don't provide a full out of the box pipeline (yet!), there are plenty of examples and documentation which walk you though the math, implementation, and other issues. If you want to read about the graphical models underlying GTSAM, see [2] Utilizing OpenCV for feature detection and association is pretty much all you really need to add to a program in order to recreate Photosynth using gtsam. I'd also you recommend KAZE features from a former post-doc out of our lab, it's state of the art and recently added OpenCV wrappers[3]. However, it's also trivial to integrate other sensors such as IMUs, GPS, lasers, etc. for full navigation problems. If you wish to know more about the actual subject, I definitely recommend Hartley and Zisserman's Multiview Geometery Book[4] [1] https://borg.cc.gatech.edu/borg/ https://borg.cc.gatech.edu/borg/ [2] http://www.cc.gatech.edu/~dellaert/pub/Dellaert06ijrr.pdf http://www.cc.gatech.edu/~dellaert/pub/Dellaert06ijrr.pdf [3] https://github.com/pablofdezalc/akaze https://github.com/pablofdezalc/akaze [4] http://www.robots.ox.ac.uk/~vgg/hzbook/index.html http://www.robots.ox.ac.uk/~vgg/hzbook/index.html
- joshu 13y ago
  do you think it would be possible and/or worth it to reconstruct a automotive road course? this is something i've always wanted to do. there's a ton of video online (though usually shot from inside the car.) the one i spend the most time at is pretty flat, though. i suspect the green hills are hard to get a match on? here is an example: http://www.youtube.com/watch?v=EyZcERAlBeE http://www.youtube.com/watch?v=EyZcERAlBeE
  - rmc 13y ago
    I often go on rides around mountainous areas on my motorbike with a video camera attached to my helmet (some videos here http://www.youtube.com/user/slashtomeu/videos http://www.youtube.com/user/slashtomeu/videos ). Since I carry a GPS as well, I've always wondered if I could reconstruct a 3D model of the landscape I ride around in.
frik 13y ago
A good place to start is the Photogammmetry forum, there you find info about photosync, bundler, PMS, etc: http://pgrammetry.com http://pgrammetry.com

by femto 13y ago

apu 13y ago

Although I don't have first hand experience with the code, a former labmate of mine (I'm at the Univ. of Washington, and he's now at Google) is one of the leading experts on this area of research, and his VisualSFM [1] tool is, I think, the best and easiest-to-use available online. Briefly, there are three main steps required to go from images to a 3d viewer like PhotoSynth: 1. Figure out where each image was shot from (the "camera pose") and get a sparse set of 3d points from the scene. These two are estimated simultaneously using bundle adjustment [2]. 2. Go from a sparse set of 3d points to a dense 3d model. This is done using a technique called Multiple View Stereo (MVS), of which the leading (open) implementations are PMVS/CMVS [3,4]. 3. Build an image-based rendering system that intelligently blends between the 3d models and images to minimize artifacts. The VisualSFM software will do steps 1 and 2. Step 3 is still quite a challenging problem, but depending on what you're doing, you could use standard 3d modeling environments to look at your data. [1] http://ccwu.me/vsfm/ http://ccwu.me/vsfm/ [2] http://en.wikipedia.org/wiki/Bundle_adjustment http://en.wikipedia.org/wiki/Bundle_adjustment [3] http://www.di.ens.fr/pmvs/ http://www.di.ens.fr/pmvs/ [4] http://www.di.ens.fr/cmvs/ http://www.di.ens.fr/cmvs/

amelim 13y ago

Sure! My lab focuses on SLAM and 3D reconstruction, especially for robotic applications. We've developed a BSD-licensed C++ (w/ MATLAB wrapper) library with specific applications towards 3D reconstruction problems such as SLAM and structure from motion. It's called GTSAM [1] We actively maintain and release new features as they are published. While we don't provide a full out of the box pipeline (yet!), there are plenty of examples and documentation which walk you though the math, implementation, and other issues. If you want to read about the graphical models underlying GTSAM, see [2] Utilizing OpenCV for feature detection and association is pretty much all you really need to add to a program in order to recreate Photosynth using gtsam. I'd also you recommend KAZE features from a former post-doc out of our lab, it's state of the art and recently added OpenCV wrappers[3]. However, it's also trivial to integrate other sensors such as IMUs, GPS, lasers, etc. for full navigation problems. If you wish to know more about the actual subject, I definitely recommend Hartley and Zisserman's Multiview Geometery Book[4] [1] https://borg.cc.gatech.edu/borg/ https://borg.cc.gatech.edu/borg/ [2] http://www.cc.gatech.edu/~dellaert/pub/Dellaert06ijrr.pdf http://www.cc.gatech.edu/~dellaert/pub/Dellaert06ijrr.pdf [3] https://github.com/pablofdezalc/akaze https://github.com/pablofdezalc/akaze [4] http://www.robots.ox.ac.uk/~vgg/hzbook/index.html http://www.robots.ox.ac.uk/~vgg/hzbook/index.html

joshu 13y ago
do you think it would be possible and/or worth it to reconstruct a automotive road course? this is something i've always wanted to do. there's a ton of video online (though usually shot from inside the car.) the one i spend the most time at is pretty flat, though. i suspect the green hills are hard to get a match on? here is an example: http://www.youtube.com/watch?v=EyZcERAlBeE http://www.youtube.com/watch?v=EyZcERAlBeE
- rmc 13y ago
  I often go on rides around mountainous areas on my motorbike with a video camera attached to my helmet (some videos here http://www.youtube.com/user/slashtomeu/videos http://www.youtube.com/user/slashtomeu/videos ). Since I carry a GPS as well, I've always wondered if I could reconstruct a 3D model of the landscape I ride around in.

frik 13y ago

A good place to start is the Photogammmetry forum, there you find info about photosync, bundler, PMS, etc: http://pgrammetry.com http://pgrammetry.com