6 ms·

AIs are still not able to understand negations. Try "ramen without egg" or "ramen with no egg" and it will show ramen WITH egg. Or "man without striped shirt"

AIs are still not able to understand negations. Try "ramen without egg" or "ramen with no egg" and it will show ramen WITH egg. Or "man without striped shirt" will give "man WITH striped shirt"

pyinstallwoes 2y ago
That’s what negative prompt is for. Stable diffusion also isn’t like llms. LLMs certainly understand negation.
- executesorder66 2y ago
  I did mean AI's in general, so I have edited my original post. > That’s what negative prompt is for. This is what I mean by it "not understanding negations" You need whole separate prompt, just to say you want e.g. "ramen without egg" instead of just saying it in a single prompt that it understands.
  - spywaregorilla 2y ago
    You are not correct. LLMs understand "ramen without egg". Image gen models generally do not as this is not how images are described. If you want to generate ramen without egg, you'll want _negative weighted_ prompts. "eating ramen, (egg:-1)"
    - Dwedit 2y ago
      Negative weighted tokens don't do what you think they do. Sometimes they act like a negative prompt, other times they don't. Likewise, zero-weight tokens don't act like the token is absent from the prompt.
      - spywaregorilla 2y ago
        They generally do. It is difficult to differentiate "negative tokens don't do that" and "prompt adherence is shaky in general". It's fully possible that the image model draws eggs in ramen but it doesn't know that the egg is an egg and therefore any attempts to interact with it via the egg token are futile. Generally speaking though thing:-1 should reduce the presence of thing for well understood concepts. It's a better tool on second pass alterations of an image.
  - viraptor 2y ago
    You're not correct about AIs in general. Both chat LLM models and sentence embeddings can handle negation just fine. (Ask any chat "what clothes would a person wear if they weren't wearing a hat") Here it was simply not trained for that purpose. Maybe it wasn't worth it, maybe the creators thought that the negative prompt is enough, maybe the time was better spent on other examples. They way, it's not AIs in general, and it's not a tech limitation.
viraptor 2y ago
It's not trained for it, because that use case is handled differently. It would be mostly a waste of time to train the concept compared to other things you want to achieve. Instead you put things you don't want in the negative prompt. This example doesn't expose the option, but you can try it here for a different model: https://huggingface.co/spaces/gokaygokay/Kolors https://huggingface.co/spaces/gokaygokay/Kolors Set the seed to 0 and prompt to "man in a loud shirt" - you get flowers. Sweet the negative prompt to "floral shirt" - no not flowers. Sentence processors can definitely understand negation, (any non-trivial LLM can) but it would be a waste of time to train that in the image generators -vs- making other ideas better.
- gowld 2y ago
  Why can't an imagen generate run a tiny little automatic text-to-text rewrite first, to apply these special linguistic rules?
  - spywaregorilla 2y ago
    some do. but generally people just learn to use the tools.
  - viraptor 2y ago
    Apps / interfaces to those models can totally add that. But it's not necessary to add that to the model itself.
GaggiX 2y ago
>AIs are still not able to understand negations. AIs are able to understand negations, just ask an LLM a question. Text-to-image models are the ones that struggle the most with this, they usually do not have a very nuanced understanding of text.

AIs are still not able to understand negations. Try "ramen without egg" or "ramen with no egg" and it will show ramen WITH egg. Or "man without striped shirt" will give "man WITH striped shirt"

pyinstallwoes 2y ago

That’s what negative prompt is for. Stable diffusion also isn’t like llms. LLMs certainly understand negation.

executesorder66 2y ago
I did mean AI's in general, so I have edited my original post. > That’s what negative prompt is for. This is what I mean by it "not understanding negations" You need whole separate prompt, just to say you want e.g. "ramen without egg" instead of just saying it in a single prompt that it understands.
- spywaregorilla 2y ago
  You are not correct. LLMs understand "ramen without egg". Image gen models generally do not as this is not how images are described. If you want to generate ramen without egg, you'll want _negative weighted_ prompts. "eating ramen, (egg:-1)"
  - Dwedit 2y ago
    Negative weighted tokens don't do what you think they do. Sometimes they act like a negative prompt, other times they don't. Likewise, zero-weight tokens don't act like the token is absent from the prompt.
    - spywaregorilla 2y ago
      They generally do. It is difficult to differentiate "negative tokens don't do that" and "prompt adherence is shaky in general". It's fully possible that the image model draws eggs in ramen but it doesn't know that the egg is an egg and therefore any attempts to interact with it via the egg token are futile. Generally speaking though thing:-1 should reduce the presence of thing for well understood concepts. It's a better tool on second pass alterations of an image.
- viraptor 2y ago
  You're not correct about AIs in general. Both chat LLM models and sentence embeddings can handle negation just fine. (Ask any chat "what clothes would a person wear if they weren't wearing a hat") Here it was simply not trained for that purpose. Maybe it wasn't worth it, maybe the creators thought that the negative prompt is enough, maybe the time was better spent on other examples. They way, it's not AIs in general, and it's not a tech limitation.

viraptor 2y ago

It's not trained for it, because that use case is handled differently. It would be mostly a waste of time to train the concept compared to other things you want to achieve. Instead you put things you don't want in the negative prompt. This example doesn't expose the option, but you can try it here for a different model: https://huggingface.co/spaces/gokaygokay/Kolors https://huggingface.co/spaces/gokaygokay/Kolors Set the seed to 0 and prompt to "man in a loud shirt" - you get flowers. Sweet the negative prompt to "floral shirt" - no not flowers. Sentence processors can definitely understand negation, (any non-trivial LLM can) but it would be a waste of time to train that in the image generators -vs- making other ideas better.

gowld 2y ago
Why can't an imagen generate run a tiny little automatic text-to-text rewrite first, to apply these special linguistic rules?
- spywaregorilla 2y ago
  some do. but generally people just learn to use the tools.
- viraptor 2y ago
  Apps / interfaces to those models can totally add that. But it's not necessary to add that to the model itself.

GaggiX 2y ago

>AIs are still not able to understand negations. AIs are able to understand negations, just ask an LLM a question. Text-to-image models are the ones that struggle the most with this, they usually do not have a very nuanced understanding of text.