4 ms·
The otter examples highlight something you can't control using these things: the 'eats shoots and leaves' phenomenon. The prompt was "A cute otter in a rainbow
by JoeAltmaier 4y ago
The otter examples highlight something you can't control using these things: the 'eats shoots and leaves' phenomenon.
The prompt was "A cute otter in a rainbow whirlpool holding shells, watercolor"
Seems like the otter should be holding shells, the way a normal human parses it.
The tool showed the otter 'in holding-shells', which are shells that hold otters apparently. Also some random shells strewn about, as the technique is sensitive to spurious detail sprouting up from single words.
Until the tool permits some kind of syntactic diagramming or so forth, we'll not be able to control for this.
Just the other day here, I saw a picture of a fork and some plastic mushrooms. The prompt was 'plastic eating mushrooms' which was ambiguous even to humans. The tool chose to illustrate the subclass of mushrooms 'eating-mushrooms' (as opposed to poison mushrooms or decorative mushrooms I suppose) made of plastic.
When we're playing around this can seem whimsical and artistic. But a graphic designer might want some semblance of control over the process.
Not sure how a solution would work.
- CuriouslyC 4y agoGraphic designers lean on img2img in their workflows more than txt2img, as that gives you the control you speak of.
- rytill 4y agoDo you know of examples of graphic designers who have shared their img2img workflows online?
- CuriouslyC 4y agoHang out in reddit.com/r/stablediffusion, people have posted workflows quite a bit.
- CrazyStat 4y agoThis is the compositionality problem--the language model sometimes doesn't quite know how to put the words together. Better language models will help in the future; in the mean time you can give it a helping hand by prompt engineering or using img2img.
- UncleEntity 4y agoMy favorite is when you do “<whatever> bla, bla, bla, wearing a t-shirt by <artist>“ and it gives you an image of <whatever> wearing a t-shirt with a print in the style of the artist. Which adds extra dimensions to play with so isn’t all that bad.
- overengineer 4y agoThere must be a way to disambiguity of the prompt by adding some negative factor to clip guidance penalizing possible misunderstandings of the prompt maybe
- nanonomad 4y agohttps://energy-based-model.github.io/Compositional-Visual-Generation-with-Composable-Diffusion-Models/ https://energy-based-model.github.io/Compositional-Visual-Ge... " Compositional generation. Our method can compose multiple diffusion models during inference and generate images containing all the concepts described in the inputs without further training"