3 ms·

What am I looking at?

bestest 3y ago
basically this: https://github.com/graphdeco-inria/gaussian-splatting https://github.com/graphdeco-inria/gaussian-splatting — a somewhat different approach at rendering 3d scenes.
- underlipton 3y ago
  A thought occurred to me: is this similar to how Media Molecule's Dreams on the PS4/5 renders its scenes?
  - FinalDestiny 3y ago
    > Media Molecule “Dreams” has a splat-based renderer (I think the shipped version is not purely splat-based but a combination of several techniques). From: https://aras-p.info/blog/2023/09/05/Gaussian-Splatting-is-pretty-cool/ https://aras-p.info/blog/2023/09/05/Gaussian-Splatting-is-pr... Good eye
    - bluescrn 3y ago
      Interesting video here (from 2015) on the development of the Dreams rendering tech: https://www.youtube.com/watch?v=u9KNtnCZDMI https://www.youtube.com/watch?v=u9KNtnCZDMI
    - pixelpoet 3y ago
      Also maybe worth mentioning that the 2nd author of the InstantNGP paper is also a cofounder of and lead tech guy at Media Molecule, Alex Evans. I've been a huge fan of his work since 90s demoscene and briefly got to work with him at Lionhead Studios, guy's a legit genius.
KaiserPro 3y ago
Gaussian splatting is a fancy word for pointcloud but with coloured shapes instead of points. Its been around for ages, but It was never used because if you have a million points in a point cloud, you'd need to artistically manipulate a million points. Its like 3d hair, its pretty simple, just render a billion hairs, but in practice its hard to make it look good. Here we tell a machine learning model to adjust the angle, colour, shape and size of a million primitives (ie a square, circle, triangle etc.) so that it looks like a the photos we provide.
- Geee 3y ago
  It's a little bit more than that. Gaussians are view-dependent, which means that they can capture the full radiance field of the scene, rather than just the color and geometry of the objects. All the light bouncing around from different objects can be reproduced, including reflections etc. See the reflections here: https://www.youtube.com/watch?v=mD0oBE9LJTQ https://www.youtube.com/watch?v=mD0oBE9LJTQ This is also pretty good, but more subtle: https://www.youtube.com/watch?v=tJTbEoxxj0U https://www.youtube.com/watch?v=tJTbEoxxj0U
  - ath92 3y ago
    This implementation does not support view-dependence though (mentioned in the readme)
  - KaiserPro 3y ago
    > Gaussians are view-dependent, indeed, but that's just adding view dependent points.
- deleted 3y ago
  [deleted]
- dclowd9901 3y ago
  My initial understanding is these scenes can’t be made dynamic (animated, physically responsive). Is that correct?

basically this: https://github.com/graphdeco-inria/gaussian-splatting https://github.com/graphdeco-inria/gaussian-splatting — a somewhat different approach at rendering 3d scenes.

underlipton 3y ago
A thought occurred to me: is this similar to how Media Molecule's Dreams on the PS4/5 renders its scenes?
- FinalDestiny 3y ago
  > Media Molecule “Dreams” has a splat-based renderer (I think the shipped version is not purely splat-based but a combination of several techniques). From: https://aras-p.info/blog/2023/09/05/Gaussian-Splatting-is-pretty-cool/ https://aras-p.info/blog/2023/09/05/Gaussian-Splatting-is-pr... Good eye
  - bluescrn 3y ago
    Interesting video here (from 2015) on the development of the Dreams rendering tech: https://www.youtube.com/watch?v=u9KNtnCZDMI https://www.youtube.com/watch?v=u9KNtnCZDMI
  - pixelpoet 3y ago
    Also maybe worth mentioning that the 2nd author of the InstantNGP paper is also a cofounder of and lead tech guy at Media Molecule, Alex Evans. I've been a huge fan of his work since 90s demoscene and briefly got to work with him at Lionhead Studios, guy's a legit genius.

KaiserPro 3y ago

Gaussian splatting is a fancy word for pointcloud but with coloured shapes instead of points. Its been around for ages, but It was never used because if you have a million points in a point cloud, you'd need to artistically manipulate a million points. Its like 3d hair, its pretty simple, just render a billion hairs, but in practice its hard to make it look good. Here we tell a machine learning model to adjust the angle, colour, shape and size of a million primitives (ie a square, circle, triangle etc.) so that it looks like a the photos we provide.

Geee 3y ago
It's a little bit more than that. Gaussians are view-dependent, which means that they can capture the full radiance field of the scene, rather than just the color and geometry of the objects. All the light bouncing around from different objects can be reproduced, including reflections etc. See the reflections here: https://www.youtube.com/watch?v=mD0oBE9LJTQ https://www.youtube.com/watch?v=mD0oBE9LJTQ This is also pretty good, but more subtle: https://www.youtube.com/watch?v=tJTbEoxxj0U https://www.youtube.com/watch?v=tJTbEoxxj0U
- ath92 3y ago
  This implementation does not support view-dependence though (mentioned in the readme)
- KaiserPro 3y ago
  > Gaussians are view-dependent, indeed, but that's just adding view dependent points.
deleted 3y ago
[deleted]
dclowd9901 3y ago
My initial understanding is these scenes can’t be made dynamic (animated, physically responsive). Is that correct?