4 ms·
The solution is to use a multi-camera array (not just stereo) to capture as much of the light field as possible and then re-synthesize stereoscopic video from t
by GrantS 13y ago
The solution is to use a multi-camera array (not just stereo) to capture as much of the light field as possible and then re-synthesize stereoscopic video from the user's viewpoint by grabbing pixels located along the appropriate rays. Then you can have free head movement within the volume of space from which you captured data. Doing this for video is obviously more challenging than for a static scene, but here are the research papers from 15 years ago at Microsoft Research on doing this entire pipeline for static scenes (both PDFs).
VIRTUAL REALITY USING THE CONCENTRIC MOSAIC: CONSTRUCTION, RENDERING AND DATA COMPRESSION. Shum, Ng, Chan, 2000.
http://charlesneedham.com/pubs/68824/virtual_reality_concentric_mosaic.pdf http://charlesneedham.com/pubs/68824/virtual_reality_concent...
Rendering with Concentric Mosaics. Shum, He. 1999.
http://research.microsoft.com/pubs/68819/rendering_concentric_mosaics.pdf http://research.microsoft.com/pubs/68819/rendering_concentri...
- modeless 13y agoYou're right, I guess "always" is an overstatement. A light field 360 degree video camera would do the job, but the technology is a long way off, much farther than good VR and probably farther than good augmented reality too. The amount of data you'd need to capture and store for such a video would be staggering.
- goatforce5 13y agoThese guys are doing it: http://biglook360.com/ http://biglook360.com/
- modeless 13y ago360 degree video cameras exist. 360 degree light field video cameras with a capture range big enough to support stereo rendering do not exist (yet). The two are not comparable at all.
- tdpqfkhg 13y agoFrom what I've seen. Stereoscopic 3D 360 degree camera's are pretty much impossible to do. At least not without a gigantic compound eye like camera. Even then the problem of rendering in real time is even harder. So streaming with that kind of Camera will never work. Not sure if it's possible to make a stereoscopic reflection based camera either. The distortion would make it worthless and the 3D effectively would be minimal. Has anyone tried to make a globe like camera sensor? Everything I've seen it impossible or not actually suitable for VR.
- deleted 13y ago[deleted]
- derefr 13y agoSo, basically, Microsoft Research's Photosynth tech, but where the input is a bunch of videos instead of a bunch of photos, and the result is an "animated" 3D model of the environment instead of a static one?
- GrantS 13y agoIn some ways, yes, though while the approach you describe has been explored ( http://www.inf.ethz.ch/personal/ataneja/ModelingDynamicScenes.pdf http://www.inf.ethz.ch/personal/ataneja/ModelingDynamicScene... ), that's not quite what concentric mosaics and other light field approaches are doing. One strength of light field methods is that there is no need to model the scene. (Photosynth, on the other hand, spends much of its computational budget determining the 3D position of sparse feature points and the cameras observing them.)