4 ms·

Wow. Some questions: Take for instance the fulllivingroom demo. (I prefer fps mode.) 1) How many images are input? 2) How long does it take to compute thes

Wow. Some questions: Take for instance the fulllivingroom demo. (I prefer fps mode.) 1) How many images are input? 2) How long does it take to compute these models? 3) How long does it take to prepare these models for this browser, with all levels, etc? 4) Have you tried this in VR yet?

vyrotek 3y ago
Not exactly what you asked for. But I recently came across this VR example using Gaussian Splatting instead. Exciting times. https://twitter.com/gracia_vr/status/1731731549886787634 https://twitter.com/gracia_vr/status/1731731549886787634 https://www.gracia.ai https://www.gracia.ai
duckworthd 3y ago
Glad you liked our work! 1) Around 100-150 if memory serves. This scene is part of the mip-NeRF 360 benchmark, which you can download from the corresponding project website: https://jonbarron.info/mipnerf360/ https://jonbarron.info/mipnerf360/ 2) Between 12 and 48 hours, depending on the scene. We train on 8x V100s or 16x A100s. 3) The time for preparing assets is included in 2). I don't have a breakdown for you, but it's something like 50/50. 4) Nope! A keen hacker might be able to do this themselves by editing the JavaScript code. Open your browser's DevTools and have a look -- the code is all there!
- dougmwne 3y ago
  Do you need position data to go along with the photos or just the photos? For VR, there’s going to be some very weird depth data from those reflections, but maybe they would not be so bad when you are in headset.
  - duckworthd 3y ago
    > Do you need position data to go along with the photos or just the photos? Short answer: Yes. Long answer: Yes, but it can typically be derived from images. Structure-from-motion methods are typically used to derive lens and position information for each photo in the training set. These are then used by Zip-NeRF (our teacher) and SMERF (our model) to train a model.
- duckworthd 3y ago
  Update: Code for the web viewer is here, https://github.com/smerf-3d/smerf-3d.github.io/blob/main/viewer/index.html https://github.com/smerf-3d/smerf-3d.github.io/blob/main/vie...
  - nh2 3y ago
    What is the license? The repo doesn't say.
    - duckworthd 3y ago
      Oops, I need to update the license files. Our code is released under the Apache 2.0 license, as in this repo: https://github.com/google-research/google-research/blob/master/LICENSE https://github.com/google-research/google-research/blob/mast...

vyrotek 3y ago

Not exactly what you asked for. But I recently came across this VR example using Gaussian Splatting instead. Exciting times. https://twitter.com/gracia_vr/status/1731731549886787634 https://twitter.com/gracia_vr/status/1731731549886787634 https://www.gracia.ai https://www.gracia.ai

duckworthd 3y ago

Glad you liked our work! 1) Around 100-150 if memory serves. This scene is part of the mip-NeRF 360 benchmark, which you can download from the corresponding project website: https://jonbarron.info/mipnerf360/ https://jonbarron.info/mipnerf360/ 2) Between 12 and 48 hours, depending on the scene. We train on 8x V100s or 16x A100s. 3) The time for preparing assets is included in 2). I don't have a breakdown for you, but it's something like 50/50. 4) Nope! A keen hacker might be able to do this themselves by editing the JavaScript code. Open your browser's DevTools and have a look -- the code is all there!

dougmwne 3y ago
Do you need position data to go along with the photos or just the photos? For VR, there’s going to be some very weird depth data from those reflections, but maybe they would not be so bad when you are in headset.
- duckworthd 3y ago
  > Do you need position data to go along with the photos or just the photos? Short answer: Yes. Long answer: Yes, but it can typically be derived from images. Structure-from-motion methods are typically used to derive lens and position information for each photo in the training set. These are then used by Zip-NeRF (our teacher) and SMERF (our model) to train a model.
duckworthd 3y ago
Update: Code for the web viewer is here, https://github.com/smerf-3d/smerf-3d.github.io/blob/main/viewer/index.html https://github.com/smerf-3d/smerf-3d.github.io/blob/main/vie...
- nh2 3y ago
  What is the license? The repo doesn't say.
  - duckworthd 3y ago
    Oops, I need to update the license files. Our code is released under the Apache 2.0 license, as in this repo: https://github.com/google-research/google-research/blob/master/LICENSE https://github.com/google-research/google-research/blob/mast...