4 ms·
It has been shown that at least still-image generators learn a 3D representation internally and uses it to bootstrap their generation. If you think about it thi
by l33tman 3y ago
It has been shown that at least still-image generators learn a 3D representation internally and uses it to bootstrap their generation. If you think about it this is the only way they can be so good at shadows and reflections, perspective and lighting etc.
- wiz21c 3y agoCould you point at some articles which explores that aspect ? It sounds very interesting. Thanks !
- l33tman 3y agoPlease see https://arxiv.org/abs/2306.05720 https://arxiv.org/abs/2306.05720 Beyond Surface Statistics: Scene Representations in a Latent Diffusion Model "... In this work, we investigate a basic interpretability question: does an LDM create and use an internal representation of simple scene geometry? Using linear probes, we find evidence that the internal activations of the LDM encode linear representations of both 3D depth data and a salient-object / background distinction. These representations appear surprisingly early in the denoising process−well before a human can easily make sense of the noisy images. ..."