2 ms·
tl;dr it allows the model to focus on different frequencies of pixel-space at different time steps. Time step zero being the low frequency/low-detail features a
by ShamelessC 4y ago
tl;dr it allows the model to focus on different frequencies of pixel-space at different time steps. Time step zero being the low frequency/low-detail features and time step 1000 being a completely “filled out”, detailed image including all the high frequency features.
Doing so in discretely many steps makes the problem tractable and differentiable.
Gross oversimplification, but that’s the gist.
- Natsu 4y agoDifferent frequencies? Does that mean it's doing something like an FFT?
- ShamelessC 4y agoBit of a delayed response - but no (well, not necessarily, anyways). Operating in the frequency domain is a useful technique in machine learning. With this method however, the reason that different "timesteps" amount to different frequencies is because you start with: 0: full gaussian noise 1: less gaussian noise, more generated pixels ... 998: barely any noise, mostly generated pixels 999: no noise. generation complete. It's actually the properties of gaussian noise that allow the network to learn the importance of the frequency domain indirectly. When there is a high frequency of noise, you can expect a low frequency in pixel space. When there is less noise, you can similarly expect the pixel space to now include both low _and_ high frequency details. By the end of denoising, the full range is meant to be learned (ideally, anyway). Having said that - they may also use a conversion to fourier space - I'm aware that at least nvidia does something like this in StyleGAN3.
- Natsu 4y agoWhy does the noise being Gaussian matter? Would it even work with other types of noise like Perlin or white noise?