I would like to see something like PhotoSynth applied to video. PhotoSynth is a program that infers three dimensional objects from two dimensional photographs.

I would like to see something like PhotoSynth applied to video. PhotoSynth is a program that infers three dimensional objects from two dimensional photographs. In particular I'd like to see this applied to public events, like sports. I'd love to re-create, say, a music show or a live football game in 3D based on livestream video coming from people's phones.