3 ms·
Note that the input here is not any arbitrary video, but a radiance field, a type of data structure that's halfway between photos and a 3D model: "The radiance
by pavlov 4y ago
Note that the input here is not any arbitrary video, but a radiance field, a type of data structure that's halfway between photos and a 3D model:
"The radiance field representation can be seen as a 5D function that maps any 3D location x and viewing direction d to volume density σ and RGB color c."
The radiance field is typically reconstructed from multiple photos of a scene, which implies that either the subject must be static or you need a fancy camera rig to capture the views simultaneously.
We don't have any content pipelines for creating and editing radiance fields, so movies produced with this approach are a ways off. But one could imagine AR glasses in a couple of years that capture the different views as you move around, then the radiance field is built in the cloud and you can create artistic re-renders of what you just saw.
- robocat 4y ago> We don't have any content pipelines for creating and editing radiance fields I presume radiance fields can be generated from computer graphic scenes? It would be awesome to choose your own style while playing a video game! Real-time generation is unlikely, but game replay videos in a surreal style would be trés cool.
- pavlov 4y agoYes, but it's an extremely expensive representation for rendering. For stylization, I'd imagine you could get close-enough results with a screen space technique if you can get some additional custom render pass buffers from the engine (material properties or whatever is most useful for the specific look). That's feasible in realtime on current hardware.