3 ms·

This reminds me of something a few friends and I tried a couple of months ago: No matter what prompt was used, neither Midjourney nor Dream Studio could genera

This reminds me of something a few friends and I tried a couple of months ago: No matter what prompt was used, neither Midjourney nor Dream Studio could generate an image of a man wearing a red suit jacket with a blue shirt. (We were trying for red suit + blue shirt + white tie... but even just the first two proved impossible.) Presumably the combination is so unusual as to run counter to the training data of the models. Likewise for a forehead with three eyes.

kleiba 4y ago
Stable Diffusion, using the prompt "red suit jacket on blue shirt": https://imgur.com/a/bwjQLxv https://imgur.com/a/bwjQLxv Further fine-tuning would certainly improve the final result.
- A_D_E_P_T 4y ago
  Interesting. Thanks. For some reason, at the time, it didn't work at all -- but I've double checked it right now with Midjourney & in Dream Studio, and it works now using the same old prompts. Old sticking points are being swept aside pretty fast... ETA: They still can't seem to make it with the white tie, though!
- zone411 4y ago
  Still funny how it loves six fingers per hand
  - dotancohen 4y ago
    Just goes to show how weird the people wearing blue shirts under red jackets were in the training images.
  - prox 4y ago
    Even for humans hands are very difficult! You usually make use of these: https://thumbs.dreamstime.com/z/isolation-artist-s-wooden-jointed-hand-model-30067174.jpg https://thumbs.dreamstime.com/z/isolation-artist-s-wooden-jo...
  - Insanity 4y ago
    Oh wow, I had to go back and check after reading your comment. Totally missed that at a quick glance.
  - ClassyJacket 4y ago
    On the other hand, I, too, could not draw a hand if my life depended on it.
  - SketchySeaBeast 4y ago
    And the eyes don't seem to be symmetrically placed.
Aerroon 4y ago
Generate an image you like. Open it up in GIMP, paint the shirt blue and jacket red. Feed resulting image into img2img with a low denoising strength (0.3-0.4 maybe) with the textual prompt of what you want. This can help you get combinations of things the AI doesn't easily do itself.
version_five 4y ago
An astronaut riding a horse is unusual too but it can make that. I think the reason is it doesn't understand compositionality, meaning it doesnt have any logic or "intuition" about the relationships between the colors and items (or eyes and forehead) so it fails to constrain the way they are assembled in the image.
morsch 4y ago
I tried to make Dall-E 2 draw "Santa giving a massage to a woman" (to personalize a gift certificate), and the results were bizarre. Mostly I got women giving Santa a massage, but even those had worse artifacts than usual. When I adjusted the prompt to be more explicit as to who was giving the massage and who was receiving it, the results got increasingly weird. With some finagling, I got an acceptable image, but it was much worse than I would have expected. On the other hand, the whole thing made for a funny story to tell when I was handing over the present. https://imgur.com/a/3UWLBeV https://imgur.com/a/3UWLBeV This is line art (because I wanted to print it in black and white), but the non-line-art style had the same issue. Also it's Saint Nicholas and not Santa, but I just confirmed that the issue still persists with Santa.
- londons_explore 4y ago
  I suspect Dall-E 2 has had some kind of 'negative training' for pornography... Ie. something beyond just filtering porn from the training set. Perhaps they put pornographic images through it with a reversed loss function? And A woman giving santa a massage is just too similar to a bunch of porn...
  - jtbayly 4y ago
    You've misread. Santa is supposed to be giving the massage, but instead the woman is in the generated output. If your assumption is correct about porn having lots of women massaging men, then these outputs point to positive training with porn, rather than negative.
    - morsch 4y ago
      The generated images also had almost all women wearing heels, wearing short skirts, showing cleavage, ...
BeFlatXIII 4y ago
> Likewise for a forehead with three eyes. No Shiva images were included in the training data?
phiy 4y ago
this is pretty fascinating to think about the implications. Everyone assumes that AI is going to be all powerful and scary, but these little edge cases just completely stop it in its tracks. It makes me wonder if any of the self-driving car projects have run into weird edge cases like this that which has stopped them from releasing it more seriously. I can see many types of clothing could completely throw off the cars in a crosswalk type situation. Reflections are solved with lidar, I wonder if a sudden breeze picks up a dense cloud of dust, that the car reacts very unpredictably. Its a possibility that insurance scams will evolve in the future to target these vehicles using adversarial techniques that for humans are obvious. How could we even stop that? It'd likely be a cat and mouse game.

kleiba 4y ago

Stable Diffusion, using the prompt "red suit jacket on blue shirt": https://imgur.com/a/bwjQLxv https://imgur.com/a/bwjQLxv Further fine-tuning would certainly improve the final result.

A_D_E_P_T 4y ago
Interesting. Thanks. For some reason, at the time, it didn't work at all -- but I've double checked it right now with Midjourney & in Dream Studio, and it works now using the same old prompts. Old sticking points are being swept aside pretty fast... ETA: They still can't seem to make it with the white tie, though!
zone411 4y ago
Still funny how it loves six fingers per hand
- dotancohen 4y ago
  Just goes to show how weird the people wearing blue shirts under red jackets were in the training images.
- prox 4y ago
  Even for humans hands are very difficult! You usually make use of these: https://thumbs.dreamstime.com/z/isolation-artist-s-wooden-jointed-hand-model-30067174.jpg https://thumbs.dreamstime.com/z/isolation-artist-s-wooden-jo...
- Insanity 4y ago
  Oh wow, I had to go back and check after reading your comment. Totally missed that at a quick glance.
- ClassyJacket 4y ago
  On the other hand, I, too, could not draw a hand if my life depended on it.
- SketchySeaBeast 4y ago
  And the eyes don't seem to be symmetrically placed.

Aerroon 4y ago

Generate an image you like. Open it up in GIMP, paint the shirt blue and jacket red. Feed resulting image into img2img with a low denoising strength (0.3-0.4 maybe) with the textual prompt of what you want. This can help you get combinations of things the AI doesn't easily do itself.

version_five 4y ago

An astronaut riding a horse is unusual too but it can make that. I think the reason is it doesn't understand compositionality, meaning it doesnt have any logic or "intuition" about the relationships between the colors and items (or eyes and forehead) so it fails to constrain the way they are assembled in the image.

morsch 4y ago

I tried to make Dall-E 2 draw "Santa giving a massage to a woman" (to personalize a gift certificate), and the results were bizarre. Mostly I got women giving Santa a massage, but even those had worse artifacts than usual. When I adjusted the prompt to be more explicit as to who was giving the massage and who was receiving it, the results got increasingly weird. With some finagling, I got an acceptable image, but it was much worse than I would have expected. On the other hand, the whole thing made for a funny story to tell when I was handing over the present. https://imgur.com/a/3UWLBeV https://imgur.com/a/3UWLBeV This is line art (because I wanted to print it in black and white), but the non-line-art style had the same issue. Also it's Saint Nicholas and not Santa, but I just confirmed that the issue still persists with Santa.

londons_explore 4y ago
I suspect Dall-E 2 has had some kind of 'negative training' for pornography... Ie. something beyond just filtering porn from the training set. Perhaps they put pornographic images through it with a reversed loss function? And A woman giving santa a massage is just too similar to a bunch of porn...
- jtbayly 4y ago
  You've misread. Santa is supposed to be giving the massage, but instead the woman is in the generated output. If your assumption is correct about porn having lots of women massaging men, then these outputs point to positive training with porn, rather than negative.
  - morsch 4y ago
    The generated images also had almost all women wearing heels, wearing short skirts, showing cleavage, ...

BeFlatXIII 4y ago

> Likewise for a forehead with three eyes. No Shiva images were included in the training data?

phiy 4y ago

this is pretty fascinating to think about the implications. Everyone assumes that AI is going to be all powerful and scary, but these little edge cases just completely stop it in its tracks. It makes me wonder if any of the self-driving car projects have run into weird edge cases like this that which has stopped them from releasing it more seriously. I can see many types of clothing could completely throw off the cars in a crosswalk type situation. Reflections are solved with lidar, I wonder if a sudden breeze picks up a dense cloud of dust, that the car reacts very unpredictably. Its a possibility that insurance scams will evolve in the future to target these vehicles using adversarial techniques that for humans are obvious. How could we even stop that? It'd likely be a cat and mouse game.