4 ms·
> it appears to be generating the image tokens and decoding them akin to the original DALL-E The animation is a lie. The new 4o with "native" image generating
by Taek 2y ago
> it appears to be generating the image tokens and decoding them akin to the original DALL-E
The animation is a lie. The new 4o with "native" image generating capabilities is a multi-modal model that is connected to a diffusion model. It's not generating images one token at a time, it's calling out to a multi-stage diffusion model that has upscalers.
You can ask 4o about this yourself, it seems to have a strong understanding of how the process works.
- low_tech_love 2y agoWould it seem otherwise if it was a lie?
- Taek 2y agoThere are many clues to indicate that the animation is a lie. For example, it clearly upscales the image using an external tool after the first image renders. As another example, if you ask the model about the tokens inside of its own context, it can't see any pixel tokens. A model may not have many facts about itself, but it can definitely see what is inside of its own context, and what it sees is a call to an image generation tool. Finally, and most convincingly, I can't find a single official source where OpenAI claims that the image is being generated pixel-by-pixel inside of the context window.
- throwaway314155 2y agoSorry but I think you may be mistaken if your only source is ChatGPT. It's not aware of its own creation processes beyond what is included in its system prompt.