2 ms·

I'm having a hard time finding a reference to the hardware the inference is run on. The paper mentions training was done on a single A100 GPU so I'm going to as

by axoltl 2y ago

I'm having a hard time finding a reference to the hardware the inference is run on. The paper mentions training was done on a single A100 GPU so I'm going to assume inference was run on that same platform. The 22fps result is somewhat meaningless without that information. It does feel like we're getting closer and closer to being able to synthesize novel views in realtime from a small set of images at a framerate and quality high enough for use in AR, which is an interesting concept. I'd love to be able to 'walk around' in my photo library.

tomp 2y ago
Once the Gaussian Splats are computed (whether via ML or classical optimisation), they’re very efficient to render (similar to 3D meshes used in games). High fps isn’t incredible. Having said that (I have yet to read the paper), "efficiency" probably refers to the first part (calculating the gaussians in the first place) not rendering.
- axoltl 2y ago
  You are correct. I was confusing this technique with Novel View Synthesis through diffusion (recent paper: https://arxiv.org/abs/2408.06157 https://arxiv.org/abs/2408.06157) where inference means generating frames rather than points.
- dagmx 2y ago
  They’re not “very efficient “. They have a significant amount of overdraw due to their transparency and will be a lot more inefficient if you’re only considering material-less surface representation. They’re more efficient to capture however . They’re also more constant in their render time, but meshes will easily be faster in most scenes cases, but scale worse with complexity. The “efficiency” of splats is more about the material response and capturing complexity there, than it is about geometric representation.
iforgotpassword 2y ago
> I'd love to be able to 'walk around' in my photo library. Yes this. I've been dreaming about this since I digitized my childhood photos a few years ago. There should be more than enough photos to reconstruct the entire apartment. Or my grandparents' house. Not sure though what happens if items and furniture moves around between shots. I haven't looked much into this yet and just assumed it will need a bit more time until there is a batteries included solution I can just download and run without reading ten pages of instructions and buying a GPU cluster.

by axoltl 2y ago

tomp 2y ago

Once the Gaussian Splats are computed (whether via ML or classical optimisation), they’re very efficient to render (similar to 3D meshes used in games). High fps isn’t incredible. Having said that (I have yet to read the paper), "efficiency" probably refers to the first part (calculating the gaussians in the first place) not rendering.

axoltl 2y ago
You are correct. I was confusing this technique with Novel View Synthesis through diffusion (recent paper: https://arxiv.org/abs/2408.06157 https://arxiv.org/abs/2408.06157) where inference means generating frames rather than points.
dagmx 2y ago
They’re not “very efficient “. They have a significant amount of overdraw due to their transparency and will be a lot more inefficient if you’re only considering material-less surface representation. They’re more efficient to capture however . They’re also more constant in their render time, but meshes will easily be faster in most scenes cases, but scale worse with complexity. The “efficiency” of splats is more about the material response and capturing complexity there, than it is about geometric representation.

iforgotpassword 2y ago

> I'd love to be able to 'walk around' in my photo library. Yes this. I've been dreaming about this since I digitized my childhood photos a few years ago. There should be more than enough photos to reconstruct the entire apartment. Or my grandparents' house. Not sure though what happens if items and furniture moves around between shots. I haven't looked much into this yet and just assumed it will need a bit more time until there is a batteries included solution I can just download and run without reading ten pages of instructions and buying a GPU cluster.