2 ms·
Looking at the output, which is impressive, I want to see this pipeline applied to splats. Dense point clouds lose a bunch of color and directional information
by vessenes 2y ago
Looking at the output, which is impressive, I want to see this pipeline applied to splats. Dense point clouds lose a bunch of color and directional information needed for high quality splats, but it seems easy to imagine this method would work well for splats. I wonder if the architecture could be fine tuned for this or if you’d need to retrain an entire model.
- w-m 2y agoCertainly possible, read 4.6. Finetuning for Downstream Tasks in the paper, the first subsection is "Feed-forward Novel View Synthesis". They chose to report their experiments on LVSM, which is not an explicit representation like 3D Gaussian Splatting, but they're citing two feed-forward 3DGS approaches in their state of the art listing. Should be quite exciting going forward, as fine-tuning might be possible on consumer hardware / single Desktop machines (like it is with LLMs). So I would expect a lot of experiments coming out in this space, soon-ish. If the results hold true, it'll be pretty exciting to drop slow and cumbersome COLMAP processing and scene optimization for a single forward pass that lasts a few seconds.
- vessenes 2y agoTotally agreed. An A100 is not expensive these days also, especially for finetuning.