4 ms·

> Text > Hands good god it solves the two biggest meme issues with image models in one go. Will this be the new state of the art every other model is compared

> Text > Hands good god it solves the two biggest meme issues with image models in one go. Will this be the new state of the art every other model is compared to?

Taek 4y ago
There are good reasons to believe that this will be the new state of the art by a comfortable margin. Hard to know until we can actually play with it.
gwern 4y ago
We already knew those were going to be solved by scale like using T5 instead of the really small bad text encoder SD used, because they were solved by Imagen etc.
astrange 4y ago
There's fundamental tradeoffs, as there always will be when you're compressing things into an image model. So, this is going to have new different issues. Since it's similar to Imagen, it probably can't handle long complex prompts as well, since they developed Parti afterward. Here's my question: are there any image models where, if you prompt "1+1", you get an image showing "3"?
- dragonwriter 4y ago
  > So, this is going to have new different issues. Well, yeah, its a bigger set of models (particular the language model) that takes more resources (both to train and for inference.) That’s the tradeoff. > Here’s my question: are there any image models where, if you prompt “1+1”, you get an image showing “3”? You want a t2i model that does arithmetic in the prompt, translates to it to “text displaying the number <result>”, but, also does the arithmetic wrong? Yeah, I don’t think that combination of features is in any existing model or, really, in any of the datasets used for evaluation, or otherwise on anyone’s roadmap.
  - astrange 4y ago
    Pretend I wrote 2, edit timeout closed. "Actually thinking about your prompt" is a necessary part of being able to make the prompts natural language instead of a long list of fantasy google image search terms. Useful example being "my bedroom but in a new color", but some things I've typed into Midjourney that don't work include "a really long guinea pig" (you get a regular size one), "world's best coffee" (the coffee cup gets a world on it), etc. It's just too literal. And yes, preprocessing with an LLM could do this.
  - jamilton 4y ago
    I don't think they're saying that's a goal, I think they're curious if it is the case. LLMs are bad at arithmetic, this uses a LLM to process the prompt, that class of result seems plausible.

> Text > Hands good god it solves the two biggest meme issues with image models in one go. Will this be the new state of the art every other model is compared to?

Taek 4y ago

There are good reasons to believe that this will be the new state of the art by a comfortable margin. Hard to know until we can actually play with it.

gwern 4y ago

We already knew those were going to be solved by scale like using T5 instead of the really small bad text encoder SD used, because they were solved by Imagen etc.

astrange 4y ago

There's fundamental tradeoffs, as there always will be when you're compressing things into an image model. So, this is going to have new different issues. Since it's similar to Imagen, it probably can't handle long complex prompts as well, since they developed Parti afterward. Here's my question: are there any image models where, if you prompt "1+1", you get an image showing "3"?

dragonwriter 4y ago
> So, this is going to have new different issues. Well, yeah, its a bigger set of models (particular the language model) that takes more resources (both to train and for inference.) That’s the tradeoff. > Here’s my question: are there any image models where, if you prompt “1+1”, you get an image showing “3”? You want a t2i model that does arithmetic in the prompt, translates to it to “text displaying the number <result>”, but, also does the arithmetic wrong? Yeah, I don’t think that combination of features is in any existing model or, really, in any of the datasets used for evaluation, or otherwise on anyone’s roadmap.
- astrange 4y ago
  Pretend I wrote 2, edit timeout closed. "Actually thinking about your prompt" is a necessary part of being able to make the prompts natural language instead of a long list of fantasy google image search terms. Useful example being "my bedroom but in a new color", but some things I've typed into Midjourney that don't work include "a really long guinea pig" (you get a regular size one), "world's best coffee" (the coffee cup gets a world on it), etc. It's just too literal. And yes, preprocessing with an LLM could do this.
- jamilton 4y ago
  I don't think they're saying that's a goal, I think they're curious if it is the case. LLMs are bad at arithmetic, this uses a LLM to process the prompt, that class of result seems plausible.