3 ms·

This new model is autoregression-based (similar to LLMs, token by token) rather than diffusion based, meaning that it adheres to text prompts with much higher a

This new model is autoregression-based (similar to LLMs, token by token) rather than diffusion based, meaning that it adheres to text prompts with much higher accuracy. As an example, some users (myself included) of a generative image app were trying to make a picture of person in the pouch of a kangaroo. No matter what we prompted, we couldn’t get it to work. GPT-4o did it in one shot!

yousif_123123 1y ago
It's a mix of both it feels to me as I've been testing it. For example, you can't get it to make a clock showing custom time like 3:30, or someone writing with their left hand.. And it can't do follow many instructions or do them very precisely. But it shows that this kind of architecture will be be capable of that if scaled up most likely.
- jumploops 1y ago
  These are great tests, thanks for sharing! And you seem to be right, though the only reference I can find is in one of the example images of a whiteboard posted on the announcement[0]. It shows: tokens -> [transformer] -> [diffusion] pixels hjups22 on Reddit[1] describes it as: > It's a hybrid model. The AR component generates control embeddings that then get decoded by a diffusion model. But the control embeddings are accurate enough to edit and reconstruct the images surprisingly well. [0]https://openai.com/index/introducing-4o-image-generation/ https://openai.com/index/introducing-4o-image-generation/ [1]https://www.reddit.com/r/MachineLearning/comments/1jkt42w/comment/mjyvwx5/ https://www.reddit.com/r/MachineLearning/comments/1jkt42w/co...
  - yousif_123123 1y ago
    Yes. Also, when testing low vs high, it seems the difference is mainly in the diffusion part, as the structure of the image and the instruction following ability is usually the same. Still, very exciting and for the future as well. It's still pretty expensive and slow. But moving in the right direction.
n2d4 1y ago
Source? It's much more likely that the LLM generates the latent vector which serves as an input to the diffusion model.
- jumploops 1y ago
  From the GPT-4o System Card Addendum[0]: > Unlike DALL·E, which operates as a diffusion model, 4o image generation is an autoregressive model natively embedded within ChatGPT. [0]https://cdn.openai.com/11998be9-5319-4302-bfbf-1167e093f1fb/Native_Image_Generation_System_Card.pdf https://cdn.openai.com/11998be9-5319-4302-bfbf-1167e093f1fb/...
- famouswaffles 1y ago
  Open AI said it's auto-regressive, the presentation on the app is autoregressive, it's priced auto-regressively. Why would that be more likely ? It seems like some implementation of bytedance's VAR.

yousif_123123 1y ago

It's a mix of both it feels to me as I've been testing it. For example, you can't get it to make a clock showing custom time like 3:30, or someone writing with their left hand.. And it can't do follow many instructions or do them very precisely. But it shows that this kind of architecture will be be capable of that if scaled up most likely.

jumploops 1y ago
These are great tests, thanks for sharing! And you seem to be right, though the only reference I can find is in one of the example images of a whiteboard posted on the announcement[0]. It shows: tokens -> [transformer] -> [diffusion] pixels hjups22 on Reddit[1] describes it as: > It's a hybrid model. The AR component generates control embeddings that then get decoded by a diffusion model. But the control embeddings are accurate enough to edit and reconstruct the images surprisingly well. [0]https://openai.com/index/introducing-4o-image-generation/ https://openai.com/index/introducing-4o-image-generation/ [1]https://www.reddit.com/r/MachineLearning/comments/1jkt42w/comment/mjyvwx5/ https://www.reddit.com/r/MachineLearning/comments/1jkt42w/co...
- yousif_123123 1y ago
  Yes. Also, when testing low vs high, it seems the difference is mainly in the diffusion part, as the structure of the image and the instruction following ability is usually the same. Still, very exciting and for the future as well. It's still pretty expensive and slow. But moving in the right direction.

n2d4 1y ago

Source? It's much more likely that the LLM generates the latent vector which serves as an input to the diffusion model.

jumploops 1y ago
From the GPT-4o System Card Addendum[0]: > Unlike DALL·E, which operates as a diffusion model, 4o image generation is an autoregressive model natively embedded within ChatGPT. [0]https://cdn.openai.com/11998be9-5319-4302-bfbf-1167e093f1fb/Native_Image_Generation_System_Card.pdf https://cdn.openai.com/11998be9-5319-4302-bfbf-1167e093f1fb/...
famouswaffles 1y ago
Open AI said it's auto-regressive, the presentation on the app is autoregressive, it's priced auto-regressively. Why would that be more likely ? It seems like some implementation of bytedance's VAR.