9 ms·
This technique isn't actually speeding up the NeRF rendering algorithm. It bakes the NeRF back to a semi-discrete representation (Octree of Spherical Harmonics
by phire 6y ago
This technique isn't actually speeding up the NeRF rendering algorithm.
It bakes the NeRF back to a semi-discrete representation (Octree of Spherical Harmonics voxels) which can render near-identical results at interactive speeds.
The baked data is much larger than the original NeRF model (2gb vs 5mb), but they can be downsampled to 30-100mb with little loss in quality.
- arduinomancer 6y agoSo if I understand right, for the real-time version rather than querying the NeRF to compute the frame pixels on the fly, they instead use the NeRF to pre-generate 3D Voxel data representing the scene which can then be rendered in real time using more traditional voxel rendering?
- phire 6y agoYes and No. This preserves the exact lighting equation that the NeRF learned, while traditional voxel rendering is limited to traditional lighting equations. You would have a hard time voxelizing a NeRF, because you can't extract a traditional lighting equation out of it.
- pmayrgundter 6y agoI think this is sensitive to Hinton's work on capsules, which I believe is a more reprojectible primitive. Maybe you can coax a voxel
- pmayrgundter 6y agoThe point about spherical harmonics hits home. You could sample the different harmonics with a probabilistic scattering to construct a probability distribution for a signed distance function render, and use half the very solid render pipeline