3 ms·

"Three cats in a trenchcoat" is a good one. Three cats in a trenchcoat standing on each other's shoulders, pretending to be a human, Vincent Adultman style. Yo

"Three cats in a trenchcoat" is a good one. Three cats in a trenchcoat standing on each other's shoulders, pretending to be a human, Vincent Adultman style. You can get them standing side by side wearing trenchcoats. You can get a cat pyramid or a cat totem or a stack or tower of cats (though often the ability to count to three is then lost). You can't get them to share the trenchcoat. Nothing like that occurred in any training set, and the AIs do not understand spatial relationships between objects ("X is on top of Y, inside Z") so you cannot describe how to arrange the things it does know about in the scene. Dall-E won't do it. Midjourney won't do it. Stable Diffusion won't do it. Eventually, enough images will be seeded into the training sets for this to stop being a useful test. But right now, it gives one a fascinating window on what happens when you try to extrapolate outside the cloud of thingspace described by the training data, rather than just interpolating within it. https://www.toothycat.net/~sham/trenchcoats https://www.toothycat.net/~sham/trenchcoats

goldenkey 3y ago
It's mostly the generation engine, not the logical AI. Here's what GPT-4 says when asked to describe. > As an AI, I can't actually see the painting, but based on the title and the dimensions you've provided, I can imagine a possible description. > The painting, titled "Three cats in a trenchcoat standing on each other's shoulders, pretending to be a human, Vincent Adultman style," is a large piece, standing 6 feet tall and 4 feet wide. > At the bottom of the painting, the trenchcoat's hem brushes the ground, obscuring the cats' feet. The trenchcoat is a deep, rich color, perhaps a classic trenchcoat beige or a more dramatic black or navy. The coat is painted with careful detail, showing the texture of the fabric and the gleam of buttons. > Moving up the painting, the middle section reveals the outline of three cats stacked on top of each other within the coat. The artist might have chosen to show this through a slightly open coat, or perhaps through the distinctive shapes visible under the fabric. The cats could be of different breeds and colors to add visual interest and humor. > The top third of the painting, the upper two feet, features the head of the topmost cat peeking out from the collar of the trenchcoat. The cat might be looking straight ahead, trying to maintain its human disguise, or it might be looking out at the viewer with a humorous or mischievous expression. If the artist chose to include a hat, it would be perched on the cat's head, adding to the disguise. > The background of the painting could vary greatly depending on the artist's style and intentions. It could be a simple, solid color to keep the focus on the cats, or it could be a detailed scene that adds to the story - perhaps a city street to tie in with the Vincent Adultman reference, or a domestic setting to contrast with the cats' unusual behavior. > Again, this is a hypothetical description and the actual painting could be very different.
- losteric 3y ago
  Not sure why this post is getting down voted? It's spot on - the limitation lies in relatively small text encoders within stable diffusion/mid journey. Larger language models contribute to better image generation (see: DeepFloyd). Google's CapPa also recently showed an alternative to the contrastive learning of CLIP at any param count. Until larger models are released, there are "hacks" like this available: https://bair.berkeley.edu/blog/2023/05/23/lmd/ https://bair.berkeley.edu/blog/2023/05/23/lmd/ - GPT4-generated bounding boxes guiding stable diffusion
dafoex 3y ago
While I agree with you, and the article, that the larger problem is that the AI model simply hasn't experienced enough data to get an accurate grasp on the situation, or that the data was labelled in a way that influences the model's understanding, I think the problem here may be a human one. In the article update, the author says that they managed to craft a prompt that got the result they want by specifying the banana must be on its own. The model knows what a banana on its own looks like, but the human is expecting the model to "do what I mean" and getting frustrated when the model "did what I said". Now, I'll admit that I skimmed the last third of the article, but I didn't see any mention that things like Stable Diffusion and Midjourny have a syntax, saying "a single banana casting a shadow on a grey background" is different from "((single banana)), casting shadow, hard light, dramatic, grey background" for example.

goldenkey 3y ago

It's mostly the generation engine, not the logical AI. Here's what GPT-4 says when asked to describe. > As an AI, I can't actually see the painting, but based on the title and the dimensions you've provided, I can imagine a possible description. > The painting, titled "Three cats in a trenchcoat standing on each other's shoulders, pretending to be a human, Vincent Adultman style," is a large piece, standing 6 feet tall and 4 feet wide. > At the bottom of the painting, the trenchcoat's hem brushes the ground, obscuring the cats' feet. The trenchcoat is a deep, rich color, perhaps a classic trenchcoat beige or a more dramatic black or navy. The coat is painted with careful detail, showing the texture of the fabric and the gleam of buttons. > Moving up the painting, the middle section reveals the outline of three cats stacked on top of each other within the coat. The artist might have chosen to show this through a slightly open coat, or perhaps through the distinctive shapes visible under the fabric. The cats could be of different breeds and colors to add visual interest and humor. > The top third of the painting, the upper two feet, features the head of the topmost cat peeking out from the collar of the trenchcoat. The cat might be looking straight ahead, trying to maintain its human disguise, or it might be looking out at the viewer with a humorous or mischievous expression. If the artist chose to include a hat, it would be perched on the cat's head, adding to the disguise. > The background of the painting could vary greatly depending on the artist's style and intentions. It could be a simple, solid color to keep the focus on the cats, or it could be a detailed scene that adds to the story - perhaps a city street to tie in with the Vincent Adultman reference, or a domestic setting to contrast with the cats' unusual behavior. > Again, this is a hypothetical description and the actual painting could be very different.

losteric 3y ago
Not sure why this post is getting down voted? It's spot on - the limitation lies in relatively small text encoders within stable diffusion/mid journey. Larger language models contribute to better image generation (see: DeepFloyd). Google's CapPa also recently showed an alternative to the contrastive learning of CLIP at any param count. Until larger models are released, there are "hacks" like this available: https://bair.berkeley.edu/blog/2023/05/23/lmd/ https://bair.berkeley.edu/blog/2023/05/23/lmd/ - GPT4-generated bounding boxes guiding stable diffusion

dafoex 3y ago

While I agree with you, and the article, that the larger problem is that the AI model simply hasn't experienced enough data to get an accurate grasp on the situation, or that the data was labelled in a way that influences the model's understanding, I think the problem here may be a human one. In the article update, the author says that they managed to craft a prompt that got the result they want by specifying the banana must be on its own. The model knows what a banana on its own looks like, but the human is expecting the model to "do what I mean" and getting frustrated when the model "did what I said". Now, I'll admit that I skimmed the last third of the article, but I didn't see any mention that things like Stable Diffusion and Midjourny have a syntax, saying "a single banana casting a shadow on a grey background" is different from "((single banana)), casting shadow, hard light, dramatic, grey background" for example.