3 ms·
Ooh, so this can take real images and predict albedo and lighting! Please, someone use this to make relightable gaussian splatting scenes. Dynamic lighting woul
by modeless 3y ago
Ooh, so this can take real images and predict albedo and lighting! Please, someone use this to make relightable gaussian splatting scenes. Dynamic lighting would really expand the usefulness of 3D scans made from photos, and I haven't seen anyone get anything close to what I would call "good" results in that area yet.
- IshKebab 3y agoCan it definitely use real images? I imagine extracting the depth map from real images would be the most useful application if it can.
- modeless 3y ago"Fortunately, diffusion models are not only powerful image generators, their structure as image-to-image models makes it straightforward to apply to real images." It looks like they actually only tried depth/normal maps for real images, not albedo/lighting maps, but it certainly seems possible. Honestly depth maps often look impressive at first glance but typically if you actually try to use them for anything (e.g. reprojection, DoF blur) their hidden flaws instantly become super apparent, and they aren't as useful as you might think. Gaussian splatting reconstructions already do reprojection better anyway. OTOH albedo maps look weird, but relighting is extremely useful and important. I'm not excited about yet another way to generate approximate depth maps, but I am excited about relighting.