3 ms·
What I can not wrap my head around: How are these models trained? What training mechanism or model architecture provides the glue to go from human text to imag
by jcattle 2mo ago
What I can not wrap my head around: How are these models trained?
What training mechanism or model architecture provides the glue to go from human text to images?
Don't you need to have millions of really descriptively labelled images?
- mistercheph 2mo agoI found this 3b1b guest video on diffusion helpful: https://www.youtube.com/watch?v=iv-5mZ_9CPY https://www.youtube.com/watch?v=iv-5mZ_9CPY
- deleted 2mo ago[deleted]
- nucleative 2mo agoThat's exactly how they do it. There are ML models that do the reverse and output image to text, which assist quite a lot. The better the text represents the unique thing in the photo, the better the model understands what that text means.
- vonneumannstan 2mo agoShort answer yes. Slightly longer answer for older text to image models you teach them how to encode images and text into the same latent space. Then you simply do a conversion, take a text input, put it into latent space and then extract the image that latent space represents.