4 ms·
An MLP-based NeRF actually has a comparable number of parameters to plenoxels (it's not 2-3 orders of magnitude smaller). The original NeRF is 8 dense layers w
by 3dthrow 5y ago
An MLP-based NeRF actually has a comparable number of parameters to plenoxels (it's not 2-3 orders of magnitude smaller). The original NeRF is 8 dense layers with 256 channels and then this one adds another network with 6 dense layers with 64 channels, so roughly speaking 7x256x256 + 5x64x64. And remember that the voxel grid is sparse, though we don't get exact numbers here. We shouldn't be miserly with our megabytes in 2021. What concerns me is how HyperNeRF requires 64 hours of training time on 4 TPU v4s; if you want to use this for communication or entertainment, it's light-years away from interactive.
Extending plenoxels to support dynamic objects would be great future work.
- dougabug 5y agoYou can’t really compare HyperNeRFs to Plenoxels because the former does so much more. Vanilla NeRF has just under half a million weights (if we ignore the coarse network), although more recent work has shown comparable results with only four layers. To compare apples-to-apples, we should probably also apply the (hand-tuned) TVD regularization used in plenoxels. Plenoxels store 28 parameters per grid point, so at a 256^3 grid size this would be just under half a billion parameters. You don’t know which voxels you can prune at the outset (btw, DNNs can be pruned as well), but for real scenes they show sparsity reaching ~5% after 12.8k iterations. For a real 360 deg scene, because the voxelized volume is quite bounded, you also need to consider the 64 layer multi-sphere image required to capture the background. Basically, you need a fairly large GPU card to train plenoxels on any kind of real scene. The option to make this space-compute tradeoff is still useful.