4 ms·

Plenoxels

109 points by unwind 5y ago

taliesinb 5y ago
Interesting, but the video is only available at 480p for now, so it's hard to actually see any details in the demo. I assume this was uploaded very recently and YT is still processing the higher-quality versions.
- ryandamm 5y ago
  Don't hold your breath, a lot of these projects are done at what we consumers would consider low rez.
  - vanderZwan 5y ago
    If you scroll down on the page there are high definition clips of the results.
The_rationalist 5y ago
https://alexyu.net/plenoctrees/ https://alexyu.net/plenoctrees/
kingcharles 5y ago
Forgive me if I have this all wrong, but in summary it seems to be that they have created a faster algorithm for creating a 3D (built of voxels, not polygons) from a set of 2D photos of that scene. Am I right?
- h0l0cube 5y ago
  No, AFAICT it's actually building a representation of a more detailed ray-traced 3D geometry in a way that can be rendered real time from any angle and with realistic lighting
  - ryandamm 5y ago
    Per the paper it's not quite real time (15 fps), and they suggest converting to a different representation (PlenOctrees) for real time. But yes, it's far more realistic than your simple mesh-and-texture; the video shows it handles reflections and transparency. When NeRF came out (~2020?) it was a major step forward in light field quality and this technique appears to maintain that quality.
- aidenn0 5y ago
  It seems to be a faster way of calculating differentiable volumetric geometry from a scene (whether the input is computer-generated or captured from the real world). I didn't get it at all until I explored the linked NeRF site first.
- ryandamm 5y ago
  The actual representation is using voxels to index a spherical harmonic coding of scene detail (I think), and that includes transparency so the novel view synthesis accretes light along the ray length. This lets it represent transparency and other lighting effects that aren't possible with simple photogrammetry.
modeless 5y ago
So it doesn't use a neural network, but it is still optimized by gradient descent. Differentiability is the key!
aerodog 5y ago
At this point it's not "NeRF" but "DONeRF" that they should go toe to toe with. https://github.com/facebookresearch/DONERF https://github.com/facebookresearch/DONERF

109 points by unwind 5y ago

Interesting, but the video is only available at 480p for now, so it's hard to actually see any details in the demo. I assume this was uploaded very recently and YT is still processing the higher-quality versions.

ryandamm 5y ago
Don't hold your breath, a lot of these projects are done at what we consumers would consider low rez.
- vanderZwan 5y ago
  If you scroll down on the page there are high definition clips of the results.

The_rationalist 5y ago

https://alexyu.net/plenoctrees/ https://alexyu.net/plenoctrees/

kingcharles 5y ago

Forgive me if I have this all wrong, but in summary it seems to be that they have created a faster algorithm for creating a 3D (built of voxels, not polygons) from a set of 2D photos of that scene. Am I right?

h0l0cube 5y ago
No, AFAICT it's actually building a representation of a more detailed ray-traced 3D geometry in a way that can be rendered real time from any angle and with realistic lighting
- ryandamm 5y ago
  Per the paper it's not quite real time (15 fps), and they suggest converting to a different representation (PlenOctrees) for real time. But yes, it's far more realistic than your simple mesh-and-texture; the video shows it handles reflections and transparency. When NeRF came out (~2020?) it was a major step forward in light field quality and this technique appears to maintain that quality.
aidenn0 5y ago
It seems to be a faster way of calculating differentiable volumetric geometry from a scene (whether the input is computer-generated or captured from the real world). I didn't get it at all until I explored the linked NeRF site first.
ryandamm 5y ago
The actual representation is using voxels to index a spherical harmonic coding of scene detail (I think), and that includes transparency so the novel view synthesis accretes light along the ray length. This lets it represent transparency and other lighting effects that aren't possible with simple photogrammetry.

modeless 5y ago

So it doesn't use a neural network, but it is still optimized by gradient descent. Differentiability is the key!

aerodog 5y ago

At this point it's not "NeRF" but "DONeRF" that they should go toe to toe with. https://github.com/facebookresearch/DONERF https://github.com/facebookresearch/DONERF