4 ms·
There is no mesh here. Nerfs are 5d (colours are computed based on a 3D position vector + a view direction vector) fields that are rendered volumetrically. So t
by hansworst 5y ago
There is no mesh here. Nerfs are 5d (colours are computed based on a 3D position vector + a view direction vector) fields that are rendered volumetrically. So the “texture” is an integral part of the neural representation of the scene, not just an image applied to a mesh.
The cool part is that this also allows for capturing transparency, and any effects caused by lighting (including complex specular reflections) are embedded into the representation.
- soylentgraham 5y agoNitpicking, but for GP; Nerf is the internal representation, but the output doesnt have to be 2D (ray traced basically) There are examples of people outputting SDF (and by extension geometry) with nerf, and projecting original texture onto that would give some nice effects; (live volumetric works best this way) though there would be some disparity where edges/occlusion isnt perfect, so youd want to sample nerf's rgb anyway... although a lot of that is fuzzy at the edges too. A lot of incorrect transparency at edges looks great in the 2D renders (so much anti aliasing and noise!) but less good for texturing
- ath92 5y agoA NeRF is not the same as an SDF though. NGP (the paper by Nvidia linked here) can train NeRFs and SDFs, but I don't know of any straightforward way of extracting an SDF from a NeRF. And while it's true that there are methods for extracting a surface from a NeRF, achieving a high quality result can be challenging because you have to figure out what to do with regions that have low occupancy (i.e. regions that are translucent). Should you consider those regions as contained within the surface, or outside of it? Especially when dealing with things like hair, it's not obvious how to construct a surface based on a NeRF.