4 ms·

Hard to believe the original Gaussian Splatting paper is still less than three months old, given the explosion of new techniques in recent weeks. It's wild to s

Hard to believe the original Gaussian Splatting paper is still less than three months old, given the explosion of new techniques in recent weeks. It's wild to see the state of the art in environment and object capture suddenly advancing this quickly – beyond the obvious applications like real estate, I wonder what else GS will end up transforming.

bane 3y ago
To be at risk of an "AcKchYuALly". Gaussian Splatting has been around since at least the early 90s. There's even a few old games made with the technique. The paper I think your referring to made the interesting leap that a 3d radiance field could be rerendered out as a field of Gaussian splats, and that this would probably run faster in modern GPU pipelines for real-time performance. It looks like they also have the nice property of being able to be shifted around in memory quickly hence the animation property seen here.
- krasin 3y ago
  Interesting! Can you please name some of these old games made with Gaussian Splatting? I would be interested to play, to get a sense why polygons won in that round (and likely to lose in this one).
  - otteromkram 3y ago
    From another user: https://www.youtube.com/watch?v=dnOXk3QJWN8 https://www.youtube.com/watch?v=dnOXk3QJWN8
  - chaboud 3y ago
    I used additive gaussian fields (restricted by bounding regions) for this back in the late 90's for audio visualizations in a ripper/player called "Siren" (back when we actually thought we could charge money for something like that). The technique worked well on non-accelerated (CPU only) hardware of the era, with the additive approach saving the pain of needing to keep a z buffer or fragment list. Gaussian voxel reconstruction is useful in medical and GIS settings, which, if memory serves, is where Kyle Freeman from Novalogic drew on for his work on Comanche. As far as I know, that was the first commercial game with voxel rendering... It's been a bit since I played it, but the swimming jaggies make me think that it was Manhattan distance height map offset by planar traversal (kinda like Doom raycasting) or some similar trick. I don't recall any intersections or overhangs, but, to be fair, I was a middle schooler when Comanche came out. It also ran fine on my weak sauce PC. Once acceleration hit, transformation of triangles with fixed-function pipelines took over. The ability to push textured triangles with minimal per-pixel value adjustment took over. Slowly but surely we've swing back to high ALU balance (albeit via massive stream parallelism). We've shifted from heavy list/vertex transformers to giant array multiply/add processors. It's a pretty great time to be a processing nerd.
- constantlm 3y ago
  Could you point us to some examples of old games using this technique? Would be awesome to see.
  - doormatt 3y ago
    Ecstatica - https://www.youtube.com/watch?v=dnOXk3QJWN8 https://www.youtube.com/watch?v=dnOXk3QJWN8
    - drdeca 3y ago
      Is this really splatting gaussians? Or is it rendering ellipsoids?
      - corysama 3y ago
        It was just ellipsoids. I don’t know if any game specifically used Gaussians. But, the idea of splatting points, Gaussians, ellipsoids and a variety of other shapes has been around for at least 20 years. The novelty of the paper was in using the differentiability of Gaussians to enable fitting splats to incrementally match all of the target photos simultaneously. So, it’s a new way to generate the splats from photos rather than modeling them by hand.
        scheeseman486 3y ago
        PlayStation Dreams used a very similar technique.
      - gcr 3y ago
        The backgrounds are static and prerendered! There's one sphere .bmp that's scaled and stretched. It comes with a depth offset map that populates a simple z-buffer to prevent overdraw. So rendering each frame becomes just a couple hundred dozen operations!
- kelseyfrog 3y ago
  The point of Gaussian Splatting for me is that it is a learned representation. It's odd that others view it primarily as a drawing sprites. I'm curious, would you classify particle effects drawn with quads as 4D gaussian splatting too?
  - bane 3y ago
    Well, in the old days, you just put the splats in your 3d space, they weren't really sprites (in the strict sense that they didn't use dedicated sprite hardware). They're really interesting thing is that they're being used here to render the learned representation, but there's nothing particularly special about them or new or AI/ML about them. You could "model" 3d objects with the gaussians by just putting a bunch together. It was a way to produce fast rendering 3d images without using a bunch of polygons. The results back then were...left behind by other techniques. There's a massive back catalog of computer graphics work on the technique, it's usually just easiest to use the search tools and search back for all dates leading up to say...2021 and you'll find tons of normal old stuff like CS 302 - Computer Graphics courseware slides or whatever on the technique. https://www.google.com/search?q=gaussian+splat+-site%3Apinterest.*&sca_esv=573896841&hl=en&sxsrf=AM9HkKmpdTVJGgCrENkavpKV_rTlshXClA%3A1697499680577&source=lnt&tbs=cdr%3A1%2Ccd_min%3A12%2F1%2F1989%2Ccd_max%3A2%2F1%2F2021&tbm= https://www.google.com/search?q=gaussian+splat+-site%3Apinte...
    - ezconnect 3y ago
      Being old and seeing the new generations amazed by the reapplication of what was discovered and used decades ago in a novel way amazes me.
- blovescoffee 3y ago
  If you want to be pedantic, the paper made the leap it did because of differentiable rendering which necessarily needs a differentiable representation of primitives - so they use Gaussians. It’s entirely novel and set in a nascent field (neural rendering). Gaussians happen to be further representable as easily rasterized primitives. Though some considerable work was put into making this performant. Everyone who keeps saying this has been around since the 90s is missing the context of the very modern differentiable rendering literature.
  - deleted 3y ago
    [deleted]
- verytrivial 3y ago
  I remember a 4k demo that used translucent triangles (I think? my brain is showing me circles, so perhaps a fixed set of sizes and fast blit with alpha.) This created moving volumetric light and shadows around some geometric shapes, some pillars I think. Very smeary/ghostly with overdrawn shapes, but the effect was startling given it was on a 486. It didn't render full frames, but moved the model and just kept splatting.
deleted 3y ago
[deleted]
atonalfreerider 3y ago
This one was released today as well. Works out of the box: https://github.com/JonathonLuiten/Dynamic3DGaussians https://github.com/JonathonLuiten/Dynamic3DGaussians
jacobgorm 3y ago
We used to call this technique "vector balls" on the Amiga, here is one famous example: https://www.youtube.com/watch?v=gjKkUTlhIek https://www.youtube.com/watch?v=gjKkUTlhIek . I remember implementing it myself for an unreleased demo. I realize that a lot has happened since, but this is likely where it all started :)

bane 3y ago

To be at risk of an "AcKchYuALly". Gaussian Splatting has been around since at least the early 90s. There's even a few old games made with the technique. The paper I think your referring to made the interesting leap that a 3d radiance field could be rerendered out as a field of Gaussian splats, and that this would probably run faster in modern GPU pipelines for real-time performance. It looks like they also have the nice property of being able to be shifted around in memory quickly hence the animation property seen here.

krasin 3y ago
Interesting! Can you please name some of these old games made with Gaussian Splatting? I would be interested to play, to get a sense why polygons won in that round (and likely to lose in this one).
- otteromkram 3y ago
  From another user: https://www.youtube.com/watch?v=dnOXk3QJWN8 https://www.youtube.com/watch?v=dnOXk3QJWN8
- chaboud 3y ago
  I used additive gaussian fields (restricted by bounding regions) for this back in the late 90's for audio visualizations in a ripper/player called "Siren" (back when we actually thought we could charge money for something like that). The technique worked well on non-accelerated (CPU only) hardware of the era, with the additive approach saving the pain of needing to keep a z buffer or fragment list. Gaussian voxel reconstruction is useful in medical and GIS settings, which, if memory serves, is where Kyle Freeman from Novalogic drew on for his work on Comanche. As far as I know, that was the first commercial game with voxel rendering... It's been a bit since I played it, but the swimming jaggies make me think that it was Manhattan distance height map offset by planar traversal (kinda like Doom raycasting) or some similar trick. I don't recall any intersections or overhangs, but, to be fair, I was a middle schooler when Comanche came out. It also ran fine on my weak sauce PC. Once acceleration hit, transformation of triangles with fixed-function pipelines took over. The ability to push textured triangles with minimal per-pixel value adjustment took over. Slowly but surely we've swing back to high ALU balance (albeit via massive stream parallelism). We've shifted from heavy list/vertex transformers to giant array multiply/add processors. It's a pretty great time to be a processing nerd.
constantlm 3y ago
Could you point us to some examples of old games using this technique? Would be awesome to see.
- doormatt 3y ago
  Ecstatica - https://www.youtube.com/watch?v=dnOXk3QJWN8 https://www.youtube.com/watch?v=dnOXk3QJWN8
  - drdeca 3y ago
    Is this really splatting gaussians? Or is it rendering ellipsoids?
    - corysama 3y ago
      It was just ellipsoids. I don’t know if any game specifically used Gaussians. But, the idea of splatting points, Gaussians, ellipsoids and a variety of other shapes has been around for at least 20 years. The novelty of the paper was in using the differentiability of Gaussians to enable fitting splats to incrementally match all of the target photos simultaneously. So, it’s a new way to generate the splats from photos rather than modeling them by hand.
      - scheeseman486 3y ago
        PlayStation Dreams used a very similar technique.
    - gcr 3y ago
      The backgrounds are static and prerendered! There's one sphere .bmp that's scaled and stretched. It comes with a depth offset map that populates a simple z-buffer to prevent overdraw. So rendering each frame becomes just a couple hundred dozen operations!
kelseyfrog 3y ago
The point of Gaussian Splatting for me is that it is a learned representation. It's odd that others view it primarily as a drawing sprites. I'm curious, would you classify particle effects drawn with quads as 4D gaussian splatting too?
- bane 3y ago
  Well, in the old days, you just put the splats in your 3d space, they weren't really sprites (in the strict sense that they didn't use dedicated sprite hardware). They're really interesting thing is that they're being used here to render the learned representation, but there's nothing particularly special about them or new or AI/ML about them. You could "model" 3d objects with the gaussians by just putting a bunch together. It was a way to produce fast rendering 3d images without using a bunch of polygons. The results back then were...left behind by other techniques. There's a massive back catalog of computer graphics work on the technique, it's usually just easiest to use the search tools and search back for all dates leading up to say...2021 and you'll find tons of normal old stuff like CS 302 - Computer Graphics courseware slides or whatever on the technique. https://www.google.com/search?q=gaussian+splat+-site%3Apinterest.*&sca_esv=573896841&hl=en&sxsrf=AM9HkKmpdTVJGgCrENkavpKV_rTlshXClA%3A1697499680577&source=lnt&tbs=cdr%3A1%2Ccd_min%3A12%2F1%2F1989%2Ccd_max%3A2%2F1%2F2021&tbm= https://www.google.com/search?q=gaussian+splat+-site%3Apinte...
  - ezconnect 3y ago
    Being old and seeing the new generations amazed by the reapplication of what was discovered and used decades ago in a novel way amazes me.
blovescoffee 3y ago
If you want to be pedantic, the paper made the leap it did because of differentiable rendering which necessarily needs a differentiable representation of primitives - so they use Gaussians. It’s entirely novel and set in a nascent field (neural rendering). Gaussians happen to be further representable as easily rasterized primitives. Though some considerable work was put into making this performant. Everyone who keeps saying this has been around since the 90s is missing the context of the very modern differentiable rendering literature.
- deleted 3y ago
  [deleted]
verytrivial 3y ago
I remember a 4k demo that used translucent triangles (I think? my brain is showing me circles, so perhaps a fixed set of sizes and fast blit with alpha.) This created moving volumetric light and shadows around some geometric shapes, some pillars I think. Very smeary/ghostly with overdrawn shapes, but the effect was startling given it was on a 486. It didn't render full frames, but moved the model and just kept splatting.

deleted 3y ago

[deleted]

atonalfreerider 3y ago

This one was released today as well. Works out of the box: https://github.com/JonathonLuiten/Dynamic3DGaussians https://github.com/JonathonLuiten/Dynamic3DGaussians

jacobgorm 3y ago

We used to call this technique "vector balls" on the Amiga, here is one famous example: https://www.youtube.com/watch?v=gjKkUTlhIek https://www.youtube.com/watch?v=gjKkUTlhIek . I remember implementing it myself for an unreleased demo. I realize that a lot has happened since, but this is likely where it all started :)