4 ms·
I don't think transformers or diffusion models are inherently "unsupervised", especially not the way they're used in Stable Diffusion and related models (which
by mkaic 3y ago
I don't think transformers or diffusion models are inherently "unsupervised", especially not the way they're used in Stable Diffusion and related models (which are very much trained in a supervised fashion). I agree with the rest of your points though.
- ebalit 3y agoGenerative methods have usually been considered unsupervised. You're right that conditional generation start to blur the lines though.
- n2d4 3y ago"Generative AI" is a misnomer; it's not the same kind of "generative" as the G in GAN. While you're right about GANs, diffusion models as transformers as transformers are most commonly trained with supervised learning.
- ebalit 3y agoI disagree. Diffusion models are trained to generate the probability distribution of their training dataset, like other generative models (GAN, VAE, etc). The fact that the architecture is a Transformer (or a CNN with attention like in Stable Diffusion) is orthogonal to the generative vs discriminative divide. Unsupervised is a confusing term as there is always an underlying loss being optimized and working as a supervision signal, even for good old kmeans. But generative models are generally considered to be part of unsupervised methods.
- valec 3y agoself-supervised is a better term