4 ms·
Most of these tricks don’t work on SD; they’re cargo culting from a different model (NovelAI) whose data genuinely has those keywords in it. SD is trained off t
by astrange 4y ago
Most of these tricks don’t work on SD; they’re cargo culting from a different model (NovelAI) whose data genuinely has those keywords in it. SD is trained off the whole internet so those aren’t super common captions.
- lelandfe 4y agoYeah – I tested these sorts of "bad image" negative prompts a lot in 1.5 and found they had almost no impact whatsoever. It may be different in 2.0, like the tweet author says, but it is also pretty telling that in that tweet they're using "blurry", "blurred", and "grainy" and are rendering images with heavily blurred backgrounds and obvious film grain. Specific common keywords like "amputated" may have a positive impact, though. Hard to tell. Doing apples-to-apples comparisons with negative keywords is challenging because even a single extra keyword tends to completely change the image. One thing that SD really impressed me by, though, is its understanding of symmetry. "Symmetrical composition" is an incredibly powerful phrase: https://imgur.com/a/lioJ8ak https://imgur.com/a/lioJ8ak And it does, indeed, extend to anatomy as well – "symmetrical eyes" can help a lot, while "symmetrical arms" renders people with their arms raised or outstretched.
- astrange 4y agoSymmetry is effective for compression, so that makes sense - when messing with NovelAI I actually couldn't get it to generate asymmetrical hairstyles like Lain's.
- Agentlien 4y agoI did some tests on SD 1.5 with certain challenging prompts such as gymnasts doing a handstand. Using no negative prompt they became amorphous blobs. I'm guessing because gymnasts are often in dynamic poses which are hard for SD to understand. I decided to add a negative prompt. With a bit of experimentation I realised all the "bad" had no effect. However, "blob" actually made most of the deformities go away and "amputee" did help against partial limbs being generated. Something that worked even better was replacing "gymnast" with "athletic man"/"athletic woman" in the positive prompt.
- visarga 4y agoWelcome to the latent space where you can add, subtract and operate on words like they are mathematical objects. I suppose people are going to intuitively learn how the latent space works by exercising prompts.
- Nition 4y agoI wonder how much better these models would be if they were exactly the same, but the training images all had accurate, detailed descriptions.
- sdenton4 4y agoTrain a captioning system, generate the captions, then train the image generator...
- peddling-brink 4y agoI think they work, but not in the way that people seem to be using them. Take the negative prompt "bad hands". The AI doesn't know what bad hands are, that's a human concept. But it does know what hands are, so it hides them. In the example image the hands, arms, and feet are all hidden. In theory, using the negative prompt "hands" would be just as effective. I'm not an expert, but I was given the above explanation by someone who knows a lot more than me and it makes sense.
- numpad0 4y agoDanbooru and other "booru" websites has those exact tags. Tags in booru is in direction of a flattened and deduped YOLO output but better and manually assigned, which was exploited in NovelAI. cf.[1] 1: SFW, personally can't agree with bad_hands tag https://danbooru.donmai.us/posts/5797703 https://danbooru.donmai.us/posts/5797703
- dale_glass 4y agoThe model can be taught what "bad hands" look like by feeding it some samples of that with the "bad hands" tag. And existing image archives do have pictures tagged things like "bad hands" and "bad anatomy" because actual artists do draw things wrong sometimes. I imagine that on the long term people will start making archives of AI mistakes and train the AI on those to try to make them less common.
- astrange 4y agoThat’s exactly what NovelAI did and why I mentioned it in the first place.
- something_here 4y agoThat's not true. The community was using negative prompts before NovelAI came out with their model. And personally, I've seen negative prompts make a big difference, especially when finetuning the outputs.
- astrange 4y agoI didn't say negative prompts don't work, just that specific prompt text. Which is why textual-inversion-negative-prompting works better.
- anothernewdude 4y agokeywords? These are embedding models. Clip puts those phrases into an embedding that encompasses a location in the space you want to avoid. No need for the "keywords" to be in the image dataset.
- astrange 4y agoCLIP isn’t magic. “bad anatomy” won’t work any more than “picture that isn’t a cat” does. Try it on clip-front: https://rom1504.github.io/clip-retrieval/?back=https%3A%2F%2Fknn5.laion.ai&index=laion5B&useMclip=false&query=bad+anatomy https://rom1504.github.io/clip-retrieval/?back=https%3A%2F%2...
- anothernewdude 4y agoSo the problem with that, is you're visualising the space with only points that exist in the image dataset. The language embedding has more information that comes from the language that isn't contained in images. It handles bad, and it handles anatomy. If there aren't single images that cover that - that's exactly what language embeddings solve for.