5 ms·
Direct Voxel Grid Optimization
- chroem- 4y agoWhat's the difference from regular photogrammetry?
- goodmattg 4y agoUses coordinate-based neural networks to model the scene volumetrically. However, in the case of this paper does not use an MLP to represent the scene. Instead, proposes to directly learn a voxel grid representation. For an excellent review check out Advances in Neural Rendering: https://arxiv.org/abs/2111.05849 https://arxiv.org/abs/2111.05849
- chroem- 4y ago> learn a voxel grid representation But isn't that what photogrammetry does?
- CyberDildonics 4y agoTypically it creates polygonal models with the photos used to directly texture them.
- rasz 4y agoI think photogrammetry produces point clouds
- CharlesW 4y agoYes, and then polygonal models (and other things) are built from those. For anyone who wants a more technical dive into the photogrammetry pipeline, here's a video I made for a company called Mapware for NVIDIA GTC 21: https://youtu.be/ktDVWzshR4w?t=331 https://youtu.be/ktDVWzshR4w?t=331
- Frost1x 4y agoSome techniques for downsampling point clouds use voxelgrid representations but in general you're mapping pixel data from varied images to each other in space and producing points from that to try and capture surface geometry.
- fxtentacle 4y agoso basically Agisoft Photoscan, a photogrammetry software based on casting rays through a voxel grid?
- chpatrick 4y agoThat's not how Photoscan works.
- fxtentacle 4y agoBut it does? Agisoft will first estimate depth maps and then project them into a voxel volume for extracting the high-resolution mesh. Debug logging even lists the voxel grid dimensions.
- legosexmagic 4y agoregular photogrammetry usually means searching for common features in a bunch of photos. if you find the same feature in 3 photos you can triangulate its location in 3d space. the output of this process is a point cloud which you can then process into a triangle mesh. (google structure from motion). this OTH is differentiable voxel rendering. so basically optimizing the colors of a bunch of cubes to make it look the pictures. using backpropagation just like you would do it for neural networks.
- mysterEFrank 4y agoI don't get it, the point of NeRF is to replace a voxel grid with an MLP
- versatran01 4y agoyeah, I thought the same. It seems like nerf was a bit pointless if people are going back to voxel grid.
- momojo 4y agoI realize the Plenoxels paper is already mentioned by the author, but 2 Minute Papers does a great overview of the concepts (for those wondering what makes this different from other techniques) Here's the NeRF paper (2020) https://www.matthewtancik.com/nerf https://www.matthewtancik.com/nerf And the more recent Plenoxels explainer (2022) https://www.youtube.com/watch?v=yptwRRpPEBM https://www.youtube.com/watch?v=yptwRRpPEBM The original NeRF's novelty was the neural-net used to extrapolate the final images, but the newer paper shows that that's not really necessary. My understanding is that the secret-sauce is the 5D 'plenoxel'.
- tehsauce 4y agoOn first glance it appears older methods like TensoRF and nvidias instant-nerf are far superior to this.