3 ms·
Your shoot-out site is very useful. Could I suggest adding prompts that expose common failure modes? For example, asking the models to show clocks set to a sp
by belter 1y ago
Your shoot-out site is very useful. Could I suggest adding prompts that expose common failure modes?
For example, asking the models to show clocks set to a specific time or people drawing with their left hand. I think most, if not all models, will likely display every clock with the same time...And portray subjects drawing with their right hand.
- crooked-v 1y agoAnother I would suggest is buildings with specific unusual proportions and details(e.g. "the mansion's west wing is twice the height of the right wing and has only very wide windows"). I've yet to find a model that will do that kind of thing reliably, where it seems to just fall back on the vibes of whatever painting or book cover is vaguely similar to what's described.
- droopyEyelids 1y agogenerating a simple maze for kids is also not possible yet
- vunderba 1y agoLove this one so I've added it. The concept is very easy for most GenAI models to grasp, but it requires a strong overall cohesive understanding. Rather unbelievably, OpenAI 4o managed to produce a pass. I should also add an image that is heavy with "greebles". GenAI usually lacks the fidelity for these kinds of minor details so although it adds them - they tend to fall apart at more than a cursory examination. https://en.wikipedia.org/wiki/Greeble https://en.wikipedia.org/wiki/Greeble
- vunderba 1y ago@belter / @crooked-v Thanks for the suggestions. Most of the current prompts are a result of personal images that I wanted to generate - so I'll try to add some "classic GenAI failure modes". Musical instruments such as pianos also used to be a pretty big failure point as well.
- troupo 1y agoFor personal images I often play with wooly mammoths, and most models are incapable of generating anything but textbook images. Any deviation either becomes an elephant or an abomination (bull- or bear-like monsters)