3 ms·
Whenever someone says this I like to point out that they are very often used to train the VAE and VQVAE models that LDM models use. Slowly diffusion is encroac
by radarsat1 1y ago
Whenever someone says this I like to point out that they are very often used to train the VAE and VQVAE models that LDM models use. Slowly diffusion is encroaching on its territory with 1-step models, however, and there are now alternative methods to generate rich latent spaces and decoders too, so this is changing, but I'd say up until last year most of the image generators still used an adversarial objective for the encoder-decoder training. This year, not sure..
- lukeinator42 1y agothey're also used a lot for training current TTS and audio codec models to output speech that sounds realistic.
- pilooch 1y agoExactly, for real time applications VTO, simulators,...), i.e. 60+FPS, diffusion can't be used efficiently. The gap is still there afaik. One lead has been to distill DPM into GANs, not sure this works for GANs that are small enough for real time.
- godelski 1y agoI mean it is really hard to push diffusion models down in size so that just makes the speed part hard. I'm not sure diffusion can ever truly win in the speed race, at least without additional context like breadth of generation. But isn't that the thing? The best model is only the best in a given context? I think the weirdest thing in ML has always been acting like there's an objectively better model and no context is needed.