3 ms·

I'm still very fascinated by gaussian splatting, but the issue I always run into is the need to use colmap to generate the camera positions and get the initial

by Cieric 2mo ago

I'm still very fascinated by gaussian splatting, but the issue I always run into is the need to use colmap to generate the camera positions and get the initial point cloud. Which colmap effectively still needs Nvidia for dense point clouds last I checked. I'm curious though does anyone have anything else they've tried that worked well? I've tried a few different things (most of the time getting stuck on just compiling the thing) so I'm curious if anyone has any suggestions that I should focus more heavily on getting working. I have some poor quality videos I'd like to try and use and I'm willing to put in some manual work, but it feels like it's all automatic or nothing. Thanks for any suggestions that any of you have.

corysama 2mo ago
I know there have been papers and research projects about GS without COLMAP and GS from videos. But, I don't have links handy. Best I can do is link you to https://x.com/RadianceFields https://x.com/RadianceFields and https://radiancefields.com/ https://radiancefields.com/ They have all the news about GS every day.
- Cieric 2mo ago
  Thanks, I've been exploring a lot of the papers and seeing what I can get working. Some of it really has been interesting, but I just never seem to be able to get it to work well. I might just need to throw myself at it all a bit more. Thanks for the links though, I'll check them out.
- deleted 2mo ago
  [deleted]
ghostly_s 2mo ago
TFA mentions this project has backends for : AMD GPU (AMDGPU.jl) NVIDIA GPU (CUDA.jl) MacBook GPU(Metal.jl)
- Cieric 2mo ago
  I'm sorry, but the article doesn't cover anything about the preprocessing steps that are required for 90% of gaussian splat implementations. That preprocessing step is almost always colmap which takes pictures or videos and uses SfM to figure out the camera parameters and their position in the 3d scene, it then does basic point reconstruction and dense point reconstruction. That info is what gaussian splats take in, it converts the points from a colmap project into gaussians and then optimizes them through differentiable rendering using backprop to optimize the gaussians to better match the images at the calculated positions of the camera from colmap. Looking over the project files also shows that they consume colmap projects, my question wasn't about the gaussian splat implementation, it was about the defacto standard colmap preprocessing step that is also required.
  - fragmede 2mo ago
    What hardware do you have? I've been using colmap on my MacBook. It's been working alright.
    - Cieric 2mo ago
      It's been a while since I've tried to run it, but it's a mix of Nvidia and AMD GPUs and only AMD CPUs (everything portable is all AMD though and that's where my gripes are.) I'm sure the opencl backend has gotten better over time, but it always seems to drag behind the cuda backend. At this point I wish we could just have a vulkan backend and be done with it. Thanks for bringing that up though, after a quick search that I guess I haven't done in a long time, it appears that colmap does now have a hip backend. So I guess that's going to be my weekend now.

by Cieric 2mo ago

corysama 2mo ago

I know there have been papers and research projects about GS without COLMAP and GS from videos. But, I don't have links handy. Best I can do is link you to https://x.com/RadianceFields https://x.com/RadianceFields and https://radiancefields.com/ https://radiancefields.com/ They have all the news about GS every day.

Cieric 2mo ago
Thanks, I've been exploring a lot of the papers and seeing what I can get working. Some of it really has been interesting, but I just never seem to be able to get it to work well. I might just need to throw myself at it all a bit more. Thanks for the links though, I'll check them out.
deleted 2mo ago
[deleted]

ghostly_s 2mo ago

TFA mentions this project has backends for : AMD GPU (AMDGPU.jl) NVIDIA GPU (CUDA.jl) MacBook GPU(Metal.jl)

Cieric 2mo ago
I'm sorry, but the article doesn't cover anything about the preprocessing steps that are required for 90% of gaussian splat implementations. That preprocessing step is almost always colmap which takes pictures or videos and uses SfM to figure out the camera parameters and their position in the 3d scene, it then does basic point reconstruction and dense point reconstruction. That info is what gaussian splats take in, it converts the points from a colmap project into gaussians and then optimizes them through differentiable rendering using backprop to optimize the gaussians to better match the images at the calculated positions of the camera from colmap. Looking over the project files also shows that they consume colmap projects, my question wasn't about the gaussian splat implementation, it was about the defacto standard colmap preprocessing step that is also required.
- fragmede 2mo ago
  What hardware do you have? I've been using colmap on my MacBook. It's been working alright.
  - Cieric 2mo ago
    It's been a while since I've tried to run it, but it's a mix of Nvidia and AMD GPUs and only AMD CPUs (everything portable is all AMD though and that's where my gripes are.) I'm sure the opencl backend has gotten better over time, but it always seems to drag behind the cuda backend. At this point I wish we could just have a vulkan backend and be done with it. Thanks for bringing that up though, after a quick search that I guess I haven't done in a long time, it appears that colmap does now have a hip backend. So I guess that's going to be my weekend now.