4 ms·
GSWT: Gaussian Splatting Wang Tiles
- abetusk 11mo agoThere's no link to the paper, so I can only infer, but, if I understand correctly, this is a very simple idea: take a single Gaussian splat "tile" and find a cut when two copies are placed near each other and overlapping, using dynamic programming to vary the size of overlap and where the cut should be. Have a variety of cuts to break a uniform tiling (the Wang tiles part) and now you have different tiles with different nearest neighbor constraints that you can use to tile the plane. Probably a lot of details to be worked out in how to stitch Gaussian splats together but I imagine it's pretty do-able. I think one of the problems with Gaussian splatting is generating content. You can take a static picture of something but it's hard to know how to use it for interaction. This is a way to generate 3d textured sheets, like sunflower fields, walls, caves, etc. In my opinion, great idea.
- jayd16 11mo agoYou could always make cinema quality environments in a traditional pipeline and render the splats offline for later realtime consumption.
- VikingCoder 11mo agoEverybody Wang Tiles tonight. But seriously, I didn't realize I wanted this. I was hoping to experiment with just repeating the same tile. This gives me hope that other people will make these techniques approachable.
- slater 11mo ago> Everybody Wang Tiles tonight Damn you for putting this ear worm in my head
- Jgoauh 11mo agoCrazy to see 2 of my niche interests interact. Great idea, you could extent the idea to use example based texture synthesis, such as Image Quilting https://www.merl.com/publications/docs/TR2001-17.pdf https://www.merl.com/publications/docs/TR2001-17.pdf
- wiz21c 11mo agoThe gaussian splatting never cease to amaze me. I wonder if it would be OK to proceduraly (not by LLM) generate natural worlds for video games with that...
- michaelmior 11mo agoThe in-browser demo is very cool! It's not clear from the linked page, but the GitHub repo[0] includes links to sample tile datasets that can be used for the demo. [0] https://github.com/zengyf131/gswt_renderer https://github.com/zengyf131/gswt_renderer
- jjcm 11mo agoVery cool, performance is abysmal at least on my m1 pro mac though - only getting ~2fps.
- jasonjmcghee 11mo agoIt works "fine" on my M4 - didn't check the FPS. If you hold "Space" you move faster - there were a LOT of settings in the egui pop-up that could probably help...
- jasonjmcghee 11mo agoTook a while to figure it out, but no mouse looking AFAICT: WASD for movement, IJKL for looking, Space for "a bit faster"
- araes 11mo agoTo the original authors, really needs a default tileset for the demo, with maybe a link to the files. An an progress/upload bar for when you're trying to upload a .zip. Tried, and appeared to stall out for minutes on upload with no response.
- jasonjmcghee 11mo agoThis is very cool. I wonder how well it could be combined with Wave Function Collapse (or the Nested variant) [1] https://github.com/mxgmn/WaveFunctionCollapse https://github.com/mxgmn/WaveFunctionCollapse [2] https://nyh-dolphin.github.io/en/research/n_wfc/ https://nyh-dolphin.github.io/en/research/n_wfc/
- reactordev 11mo agoCan we get a demo of this with a wind modifier added. My goodness! I can’t wait to explore virtual worlds with this kind of grass/foliage detail.
- lawlessone 11mo agoIs it possible to make them skew(the right word here ?) in some way so that they could appear to blow in a breeze?
- rallyforthesun 11mo agoMost implementations of 3d gaussian splats are static. They are based on pointclouds and not polygons. As these are captured with images and generated from them, the process has no semantic understanding of its content. There is no technical way to rig each flower or move vertices like in traditional 3d animation. It is mainly a pointcloud with no segmentation. But there are projects working on the semantic part, which could open a way to animate the detected objects individually in future.
- lawlessone 11mo agoI i wasn't thinking individual flowers, though that would be nice, but maybe whole tiles somehow
- rallyforthesun 11mo agoIt is possible to import a 3d gaussian splat into houdini and animate it there. https://m.youtube.com/watch?v=3u9SAmr61gA https://m.youtube.com/watch?v=3u9SAmr61gA
- semi-extrinsic 11mo agoFor something as simple as this, you could probably just move the splats around with (x_hat*sin(y*t) + y_hat*sin(x*t))*exp((z_l-z_h)/2*z_h) where (x_hat,y_hat) are your basis vectors in the plane, z_l is the local z coordinate (subtract the terrain modifier used to move tiles up/down) , and z_h is the height of a flower. Or if you want to be more advanced, generate some curl noise and use it as a prefactor instead of x,y inside the sin(). And include the corresponding up-down motion as the stalks are constant length.
- kfarr 11mo agoYes just like there are shaders for the same thing with mesh-based grass. This repo already has some animation abilities: https://github.com/sparkjsdev/spark?tab=readme-ov-file#features https://github.com/sparkjsdev/spark?tab=readme-ov-file#featu...
- joegibbs 11mo agoIt looks great but I think the static-ness of gaussian splats is what's holding it back. You can't easily animate them or properly relight them like with polygonal rendering, so it's hard to combine two differently lit things. If someone came up with some kind of neural relighting for them or something it would be huge.
- chii 11mo agoAs an avid player of farming simulator, this is very interesting and i can see so many potential applications for it.
- foota 11mo agoThis is neat and I don't want to be too snarky, but I had to chuckle as the specificity of "However, extending 3DGS to synthesize large-scale or infinite terrains from a single captured exemplar—remains an open challenge.".
- nullc 11mo agoMicrosoft research had some good publications on generating infinite non-repeating textures by (IIRC) markov-filling an aperiodic tile set, including creating video textures. I tried to find the examples a few months ago but the old URLs I'd bookmarked were no longer working.