4 ms·

Hi all! Several weeks ago, Nvidia released a voxel-based radiance field rendering technique called SVRaster. I thought it was an interesting alternative to Gaus

Hi all! Several weeks ago, Nvidia released a voxel-based radiance field rendering technique called SVRaster. I thought it was an interesting alternative to Gaussian Splatting, so I wanted to experiment with it and learn more about it. I've been working on a WebGL viewer to render the SVRaster Voxel scenes from the web, since the paper only comes with a CUDA-based renderer. I decided to publish the code under the MIT license. Here's the repository: https://github.com/samuelm2/svraster-webgl/ https://github.com/samuelm2/svraster-webgl/ I think SVRaster Voxel rendering has an interesting set of benefits and drawbacks compared to Gaussian Splatting, and I think it is worth more people exploring. I'm also hosting it on vid2scene.com/voxel so you can try it out without having to clone the repository. (Note: the voxel PLY file it downloads is about 50MB so you'll probably have to be on good WiFi). Right now, there's still a lot more optimizations that would make it faster. I only made the lowest-hanging fruit optimizations. I get about 60FPS on my Laptop 3080 GPU at 2k resolution, and about 10-15 FPS on my iPhone 13 Pro Max. On the github readme, there's more details about how to create your own voxel scenes that are compatible with this viewer. Since the original SVRaster code doesn't export ply, theres an extra step to convert those voxel scenes to the ply format that's readable by the WebGL viewer. If there's enough interest, I'm also considering doing a BabylonJS version of this Also, this project was made with heavy use of AI assistance ("vibe coded"). I wanted to see how it would go for something graphics related. My brief thoughts: it is super good for the boilerplate (defining/binding buffers, uniforms, etc). I was able to get simple voxel rendering within minutes / hours. But when it comes to solving the harder graphics bugs, the benefits are a lot lower. There were multiple times where it would go in the complete wrong direction and I would have to rewrite portions manually. But overall, I think it is definitely a net positive for smaller projects like this one. In a more complex graphics engine / production environment, the benefits might be less clear for now. I'm interested in what others think.

corysama 2y ago
Great work! They'd love to see this over in https://old.reddit.com/r/GaussianSplatting/ https://old.reddit.com/r/GaussianSplatting/ :) In your observation, is this technique restricted to static geometry? Or, is there a clear path to rendering a skinned animated character using SVRaster?
- samuelm2 2y ago
  Right now, its pretty restricted to relatively static geometry. You can do simple transforms like scaling, rotating, etc to groups of voxels, so maybe down the line you'd be able to animated them!
jchw 2y ago
Very interesting. I was surprised to get a relatively decent 40-60 FPS screwing around on the demo for a few minutes on my Pixel 9 in Fennec F-Droid. I am even more surprised that it performed better than an iPhone in any conditions, but who knows what exactly is playing into that. Regarding AI coding assistance... Yeah, I imagine it would be great for someone who wants to start hundreds of small projects, because it's pretty cool what you can do in a few minutes, but it really doesn't take many days before you're much better off doing most things yourself. I guess this state of affairs is relatively good for our job security, but a real bummer from the "I want to harness the power of hundreds of digital junior devs to do my bidding" perspective. Still something I'd consider solidly useful and not just a load of hype though. BTW, it seems a bit senseless to draw frames when there is no interaction since the geometry is static. Maybe it wouldn't be a bad idea to stop drawing when there are no active interactions.
- samuelm2 1y ago
  I was also surprised that my iPhone didn't do too well with it. Regular Gaussian Splatting runs pretty well on my iPhone, almost as good as my laptop (of course, my phone screen is a lot smaller so its rendering a lot fewer pixels). The fragment shader for this is very heavy compared to gaussian splatting, so maybe it has to do with that. And I agree re: constantly drawing frames vs only drawing on change, that's a good optimization to make.
Haeuserschlucht 1y ago
Used your demo. Is that blocky version the best it can do?

corysama 2y ago

Great work! They'd love to see this over in https://old.reddit.com/r/GaussianSplatting/ https://old.reddit.com/r/GaussianSplatting/ :) In your observation, is this technique restricted to static geometry? Or, is there a clear path to rendering a skinned animated character using SVRaster?

samuelm2 2y ago
Right now, its pretty restricted to relatively static geometry. You can do simple transforms like scaling, rotating, etc to groups of voxels, so maybe down the line you'd be able to animated them!

jchw 2y ago

Very interesting. I was surprised to get a relatively decent 40-60 FPS screwing around on the demo for a few minutes on my Pixel 9 in Fennec F-Droid. I am even more surprised that it performed better than an iPhone in any conditions, but who knows what exactly is playing into that. Regarding AI coding assistance... Yeah, I imagine it would be great for someone who wants to start hundreds of small projects, because it's pretty cool what you can do in a few minutes, but it really doesn't take many days before you're much better off doing most things yourself. I guess this state of affairs is relatively good for our job security, but a real bummer from the "I want to harness the power of hundreds of digital junior devs to do my bidding" perspective. Still something I'd consider solidly useful and not just a load of hype though. BTW, it seems a bit senseless to draw frames when there is no interaction since the geometry is static. Maybe it wouldn't be a bad idea to stop drawing when there are no active interactions.

samuelm2 1y ago
I was also surprised that my iPhone didn't do too well with it. Regular Gaussian Splatting runs pretty well on my iPhone, almost as good as my laptop (of course, my phone screen is a lot smaller so its rendering a lot fewer pixels). The fragment shader for this is very heavy compared to gaussian splatting, so maybe it has to do with that. And I agree re: constantly drawing frames vs only drawing on change, that's a good optimization to make.

Haeuserschlucht 1y ago

Used your demo. Is that blocky version the best it can do?