3 ms·
How might one use a Large Language Model “to generate part or all of an image”?
by MiguelX413 3y ago
How might one use a Large Language Model “to generate part or all of an image”?
- PartiallyTyped 3y agoSomething like this: Hey ChatGPT, write a prompt for midjourney to generate a realistic photo of XYZ with ABC parameters. Then plug it into Mid-journey. Technically the LLM isn't generating the image, and I agree, but I think their point is rather obvious and we need not be intentionally obtuse nor needlessly pedantic.
- seabass-labrax 3y ago'Text-to-image' systems like Stable Diffusion really are Large Language Models: an encoder such as BERT creates a mapping from text tokens to the latent space of the image generation model. As part of this training step, the system is learning the concepts of certain words and grammatical constructs. There are quite a few in-depth explanations of the whole system; here's one for instance: https://jalammar.github.io/illustrated-stable-diffusion/ https://jalammar.github.io/illustrated-stable-diffusion/