3 ms·

So I upload a pre-baked GSplat of the ground state physical space, presumably there is some kind of calibration, then I can navigate the ROS device spatially us

So I upload a pre-baked GSplat of the ground state physical space, presumably there is some kind of calibration, then I can navigate the ROS device spatially using the GSplat to reflect position details instead of, or in addition to, actual camera feeds ? Or are they producing the splats somehow on the ROS device with limited camera poses ? Whatever the case may be, I still think the human controller side is where Splats are more useful so add a VR headset into the loop and I think this could open up real opportunities for example spatial minimaps, decoupled points of view, etc.

shadygm 1y ago
Thanks for taking a look! Just to clarify, ROSplat isn’t generating the Gaussians, it’s not a SLAM algorithm or a reconstruction tool. It’s purely a visualizer that uses ROS for message passing. The idea is that if you already have a system producing Gaussians (either live or precomputed), ROSplat lets you stream and view them in real time (as the ROS messages arrive). So in your example, yes, you could upload a pre-baked GSplat, calibrate it to the robot’s frame, and use it for navigation or visualization. Or, if your ROS device is running something like SLAM, it can publish Gaussians as it goes. In both cases, ROSplat is just making them available for visualization, nothing more. And I completely agree with you on your last comment. VR Gaussians are the way to go, I know that a company Varjo is currently working on them. Not sure if there's anything else that's available tho :/
- dheera 1y ago
  I've actually been pondering using Gaussian splats for localization, I think it could be done. The idea would be looking for the pose that minimizes the MSE in density (rather than feature points or RGB similarity which are both vulnerable to lighting changes)
- jimmySixDOF 1y ago
  Varjo are good at whatever they do but also check out @gracia_vr [1] they focus on Spalts in XR and playcanvas has supersplat which lets you view immersive mode for 3DGS [2]. [1] https://www.gracia.ai/ https://www.gracia.ai/ [2] https://github.com/playcanvas/supersplat https://github.com/playcanvas/supersplat

shadygm 1y ago

Thanks for taking a look! Just to clarify, ROSplat isn’t generating the Gaussians, it’s not a SLAM algorithm or a reconstruction tool. It’s purely a visualizer that uses ROS for message passing. The idea is that if you already have a system producing Gaussians (either live or precomputed), ROSplat lets you stream and view them in real time (as the ROS messages arrive). So in your example, yes, you could upload a pre-baked GSplat, calibrate it to the robot’s frame, and use it for navigation or visualization. Or, if your ROS device is running something like SLAM, it can publish Gaussians as it goes. In both cases, ROSplat is just making them available for visualization, nothing more. And I completely agree with you on your last comment. VR Gaussians are the way to go, I know that a company Varjo is currently working on them. Not sure if there's anything else that's available tho :/

dheera 1y ago
I've actually been pondering using Gaussian splats for localization, I think it could be done. The idea would be looking for the pose that minimizes the MSE in density (rather than feature points or RGB similarity which are both vulnerable to lighting changes)
jimmySixDOF 1y ago
Varjo are good at whatever they do but also check out @gracia_vr [1] they focus on Spalts in XR and playcanvas has supersplat which lets you view immersive mode for 3DGS [2]. [1] https://www.gracia.ai/ https://www.gracia.ai/ [2] https://github.com/playcanvas/supersplat https://github.com/playcanvas/supersplat