3 ms·

Can anyone point me to some good high level introduction to Gaussian splats? E.g. what's the benefit of using splats vs traditional polygons? is it somehow eas

by folli 3y ago

Can anyone point me to some good high level introduction to Gaussian splats? E.g. what's the benefit of using splats vs traditional polygons? is it somehow easier for the neural network to create these from the 2D photos? or what's the magic behind this?

reactordev 3y ago
https://huggingface.co/blog/gaussian-splatting https://huggingface.co/blog/gaussian-splatting Think of this as a LiDAR visualization on steroids... A point cloud of information that can be used to render "The Blob".
- yetihehe 3y ago
  But the blobs can change color and opacity depending on direction you are viewing them (spherical harmonics) which allows to encode transparency and specular reflections.
  - slimbuck 3y ago
    Splats change color based on direction, but opacity is constant.
corysama 3y ago
Polygons are great because you can make a solid, closed surface. But, they are a pain because you have to maintain the connectivity between the vertices. Adding or removing polygons in a mesh is serious work. Especially doing so on a GPU. Most LOD schemes do heavy precomputation at data compilation time and have the GPU just view a sliding window into static results. Splats however are loose points. It’s harder to get a solid appearance from them. But, you can add/remove/move each point independently. Splats don’t really use neural techniques. But, they are well set up for neural techniques to work with them. Generating the splats from photos uses some sort of linear algebra optimization/back-propagation. The Gaussian aspect is required to enable that to work because gaussians are differentiable. Hopefully that means GS should fit in well with recent advancements in differentiable rendering.
karaterobot 3y ago
The article links to a page with this video embedded in it, which I found very helpful. I'm not in this field and this was the first time I'd heard of Gaussian splatting; the original article sort of assumes you're not a rando coming to the page from Hacker News, that you are sold on the idea already, and use it so much that you need to worry about optimization. https://www.youtube.com/watch?v=T_kXY43VZnk https://www.youtube.com/watch?v=T_kXY43VZnk

by folli 3y ago

reactordev 3y ago

https://huggingface.co/blog/gaussian-splatting https://huggingface.co/blog/gaussian-splatting Think of this as a LiDAR visualization on steroids... A point cloud of information that can be used to render "The Blob".

yetihehe 3y ago
But the blobs can change color and opacity depending on direction you are viewing them (spherical harmonics) which allows to encode transparency and specular reflections.
- slimbuck 3y ago
  Splats change color based on direction, but opacity is constant.

corysama 3y ago

Polygons are great because you can make a solid, closed surface. But, they are a pain because you have to maintain the connectivity between the vertices. Adding or removing polygons in a mesh is serious work. Especially doing so on a GPU. Most LOD schemes do heavy precomputation at data compilation time and have the GPU just view a sliding window into static results. Splats however are loose points. It’s harder to get a solid appearance from them. But, you can add/remove/move each point independently. Splats don’t really use neural techniques. But, they are well set up for neural techniques to work with them. Generating the splats from photos uses some sort of linear algebra optimization/back-propagation. The Gaussian aspect is required to enable that to work because gaussians are differentiable. Hopefully that means GS should fit in well with recent advancements in differentiable rendering.

karaterobot 3y ago

The article links to a page with this video embedded in it, which I found very helpful. I'm not in this field and this was the first time I'd heard of Gaussian splatting; the original article sort of assumes you're not a rando coming to the page from Hacker News, that you are sold on the idea already, and use it so much that you need to worry about optimization. https://www.youtube.com/watch?v=T_kXY43VZnk https://www.youtube.com/watch?v=T_kXY43VZnk