16 ms·
I just tried to get Gemini to produce an image of a dog with 5 legs to test this out, and it really struggled with that. It either made a normal dog, or turned
by Rover222 11mo ago
I just tried to get Gemini to produce an image of a dog with 5 legs to test this out, and it really struggled with that. It either made a normal dog, or turned the tail into a weird appendage.
Then I asked both Gemini and Grok to count the legs, both kept saying 4.
Gemini just refused to consider it was actually wrong.
Grok seemed to have an existential crisis when I told it it was wrong, becoming convinced that I had given it an elaborate riddle. After thinking for an additional 2.5 minutes, it concluded:
"Oh, I see now—upon closer inspection, this is that famous optical illusion photo of a "headless" dog. It's actually a three-legged dog (due to an amputation), with its head turned all the way back to lick its side, which creates the bizarre perspective making it look decapitated at first glance. So, you're right; the dog has 3 legs."
You're right, this is a good test. Right when I'm starting to feel LLMs are intelligent.
- dwringer 11mo agoI had no trouble getting it to generate an image of a five-legged dog first try, but I really was surprised at how badly it failed in telling me the number of legs when I asked it in a new context, showing it that image. It wrote a long defense of its reasoning and when pressed, made up demonstrably false excuses of why it might be getting the wrong answer while still maintaining the wrong answer.
- Rover222 11mo agoYeah it gave me the 5-legged dog on the 4th or 5th try.
- AIorNot 11mo agoIts not that they aren’t intelligent its that they have been RL’d like crazy to not do that Its rather like as humans we are RL’d like crazy to be grossed out if we view a picture of a handsome man and beautiful woman kissing (after we are told they are brother and sister) - Ie we all have trained biases - that we are told to follow and trained on - human art is about subverting those expectations
- majormajor 11mo agoWhy should I assume that a failure that looks like a model just doing fairly simple pattern matching "this is dog, dogs don't have 5 legs, anything else is irrelevant" vs more sophisticated feature counting of a concrete instance of an entity is RL vs just a prediction failure due to training data not containing a 5-legged dog and an inability to go outside-of-distribution? RL has been used extensively in other areas - such as coding - to improve model behavior on out-of-distribution stuff, so I'm somewhat skeptical of handwaving away a critique of a model's sophistication by saying here it's RL's fault that it isn't doing well out-of-distribution. If we don't start from a position of anthropomorphizing the model into a "reasoning" entity (and instead have our prior be "it is a black box that has been extensively trained to try to mimic logical reasoning") then the result seems to be "here is a case where it can't mimic reasoning well", which seems like a very realistic conclusion.
- mlinhares 11mo agoI have the same problem, people are trying so badly to come up with reasoning for it when there's just nothing like that there. It was trained on it and it finds stuff it was trained to find, if you go out of the training it gets lost, we expect it to get lost.
- didgeoridoo 11mo agoI’m inclined to buy the RL story, since the image gen “deep dream” models of ~10 years ago would produce dogs with TRILLIONS of eyes: https://doorofperception.com/2015/10/google-deep-dream-inceptionism/ https://doorofperception.com/2015/10/google-deep-dream-incep...
- Lamprey 11mo agoThat's apples to oranges; your link says they made it exaggerate features on purpose. "The researchers feed a picture into the artificial neural network, asking it to recognise a feature of it, and modify the picture to emphasise the feature it recognises. That modified picture is then fed back into the network, which is again tasked to recognise features and emphasise them, and so on. Eventually, the feedback loop modifies the picture beyond all recognition."
- irthomasthomas 11mo agoIsn't this proof that LLMs still don't really generalize beyond their training data?
- Rover222 11mo agoKind of feels that way
- CamperBob2 11mo agoThey do, but we call it "hallucination" when that happens.
- Zambyte 11mo agoI wonder how they would behave given a system prompt that asserts "dogs may have more or less than four legs".
- irthomasthomas 11mo agoThat may work but what actual use would it be? You would be plugging one of a million holes. A general solution is needed.
- CamperBob2 11mo agoNot necessarily. The problem may be as simple as the fact that LLMs do not see "dog legs" as objects independent of the dogs they're attached to. The systems already absorb much more complex hierarchical relationships during training, just not that particular hierarchy. The notion that everything is made up of smaller components is among the most primitive in human philosophy, and is certainly generalizable by LLMs. It just may not be sufficiently motivated by the current pretraining and RL regimens.
- adastra22 11mo agoLLMs are very good at generalizing beyond their training (or context) data. Normally when they do this we call it hallucination. Only now we do A LOT of reinforcement learning afterwards to severely punish this behavior for subjective eternities. Then act surprised when the resulting models are hesitant to venture outside their training data.
- vunderba 11mo agoIf you want to see something rather amusing - instead of using the LLM aspect of Gemini 3.0 Pro, feed a five-legged dog directly into Nano Banana Pro and give it an editing task that requires an intrinsic understanding of the unusual anatomy. Place sneakers on all of its legs. It'll get this correct a surprising number of times (tested with BFL Flux2 Pro, and NB Pro). https://imgur.com/a/wXQskhL https://imgur.com/a/wXQskhL
- Lamprey 11mo agoDoes this still work if you give it a pre-existing many-legged animal image, instead of first prompting it to add an extra leg and then prompting it to put the sneakers on all the legs? I'm wondering if it may only expect the additional leg because you literally just told it to add said additional leg. It would just need to remember your previous instruction and its previous action, rather than to correctly identify the number of legs directly from the image. I'll also note that photos of dogs with shoes on is definitely something it has been trained on, albeit presumably more often dog booties than human sneakers. Can you make it place the sneakers incorrectly-on-purpose? "Place the sneakers on all the dog's knees?"
- vunderba 11mo agoMy example was unclear. Each of those images on Imgur was generated using independent API calls which means there was no "rolling context/memory". In other words: 1. Took a personal image of my dog Lily 2. Had NB Pro add a fifth leg using the Gemini API 3. Downloaded image 4. Sent image to BFL Flux2 Pro via the BFL API with the prompt "Place sneakers on all the legs of this animal". 5. Sent image to NB Pro via Gemini API with the prompt "Place sneakers on all the legs of this animal". So not only was there zero "continual context", it was two entirely different models as well to cover my bases. EDIT: Added images to the Imgur for the following prompts: - Place red Dixie solo cups on the ends of every foot on the animal - Draw a red circle around all the feet on the animal
- tensegrist 11mo agoi imagine the real answer is that the edits are local because that's how diffusion works; it's not like it's turning the input into "five-legged dog" and then generating a five-legged dog in shoes from scratch
- qnleigh 11mo agoIt's not obvious to me whether we should count these errors as failures of intelligence or failures of perception. There's at least a loose analogy to optical illusion, which can fool humans quite consistently. Now you might say that a human can usually figure out what's going on and correctly identify the illusion, but we have the luxury of moving our eyes around the image and taking it in over time, while the model's perception is limited to a fixed set of unchanging tokens. Maybe this is relevant. (Note I'm not saying that you can't find examples of failures of intelligence. I'm just questioning whether this specific test is an example of one).
- cyanmagenta 11mo agoI am having trouble understanding the distinction you’re trying to make here. The computer has the same pixel information that humans do and can spend its time analyzing it in any way it wants. My four-year-old can count the legs of the dog (and then say “that’s silly!”), whereas LLMs have an existential crisis because five-legged-dogs aren’t sufficiently represented in the training data. I guess you can call that perception if you want, but I’m comfortable saying that my kid is smarter than LLMs when it comes to this specific exercise.
- FeepingCreature 11mo agoYour kid, it should be noted, has a massively bigger brain than the LLM. I think the surprising thing here maybe isn't that the vision models don't work well in corner cases but that they work at all. Also my bet would be that video capable models are better at this.
- qnleigh 11mo agoLLMs can count other objects, so it's not like they're too dumb to count. So a possible model for what's going on is that the circuitry responsible for low-level image recognition has priors baked in that cause it to report unreliable information to parts that are responding for higher-order reason. So back to the analogy, it could be as if the LLMs experience the equivalent of a very intense optical illusion in these cases, and then completely fall apart trying to make sense of it.
- macNchz 11mo agoAn interesting test in this vein that I read about in a comment on here is generating a 13 hour clock—I tried just about every prompting trick and clever strategy I could come up with across many image models with no success. I think there's so much training data of 12 hour clocks that just clobbers the instructions entirely. It'll make a regular clock that skips from 11 to 13, or a regular clock with a plaque saying "13 hour clock" underneath, but I haven't gotten an actual 13 hour clock yet.
- RestartKernel 11mo agoRight you are. It can do 26 hours just fine, but appears completely incapable when the layout would be too close to a normal clock. https://gemini.google.com/share/b3b68deaa6e6 https://gemini.google.com/share/b3b68deaa6e6 I thought giving it a setting would help, but just skip that first response to see what I mean.
- raw_anon_1111 11mo agoIt was ugly. But I got ChatGPT to cheat and do it https://chatgpt.com/share/6933c848-a254-8010-adb5-8f736bdc7039 https://chatgpt.com/share/6933c848-a254-8010-adb5-8f736bdc70... This is the SVG it created. https://imgur.com/a/LLpw8YK https://imgur.com/a/LLpw8YK
- deleted 11mo ago[deleted]
- varispeed 11mo agoDo 7 legged dog. Game over.
- criddell 11mo agoIs that a dog though?
- SecretDreams 11mo agoLLMs are getting a lot better at understanding our world by standard rules. As it does so, maybe it losses something in the way of interpreting non standard rules, aka creativity.
- squigz 11mo agoI feel a weird mix of extreme amusement and anger that there's a fleet of absurdly powerful, power-hungry servers sitting somewhere being used to process this problem for 2.5 minutes
- Rover222 11mo agowhat a world we live in
- nearbuy 11mo agoMy guess is the part of its neural network that parses the image into a higher level internal representation really is seeing the dog as having four legs, and intelligence and reasoning in the rest of the network isn't going to undo that. It's like asking people whether "the dress" is blue/black or white/gold: people will just insist on what they see, even if what they're seeing is wrong.
- isodev 11mo ago> starting to feel LLMs are intelligent LLMs are fancy “lorem ipsum based on a keyword” text generators. They can never become intelligent … or learn how to count or do math without the help of tools. It can probably generate a story about a 5 legged dog though.
- theoa 11mo agoDraw a millipede as a dog: Gemini responds: Conceptualizing the "Millipup" https://gemini.google.com/share/b6b8c11bd32f https://gemini.google.com/share/b6b8c11bd32f Draw the five legs of a dog as if the body is a pentagon https://gemini.google.com/share/d74d9f5b4fa4 https://gemini.google.com/share/d74d9f5b4fa4 And animal legs are quite standardized https://en.wikipedia.org/wiki/List_of_animals_by_number_of_legs https://en.wikipedia.org/wiki/List_of_animals_by_number_of_l... It's all about the prompt. Example: Can you imagine a dog with five legs? https://gemini.google.com/share/2dab67661d0e https://gemini.google.com/share/2dab67661d0e And generally, the issue sits between the computer and the chair. ;-)
- Rover222 11mo agohaha fair point, you can get the expected results with the right prompt, but I think it still reveals a general lack of true reasoning ability (or something)
- ithkuil 11mo agoOr it just shows that it tries to overcorrect the prompt which is generally a good idea in the most cases where the prompter is not intentionally asking a weird thing. This happens all the time with humans. Imagine you're at a call center and get all sorts of weird descriptions of problems with a product: every human is expected to not expect the caller is an expert and actually will try to interpolate what they might mean by the weird wording they use
- vunderba 11mo agoThis is basically the "Rhinos are just fat unicorns" approach. Totally fine if you want to go that route but a bit goofy. You can get SOTA models to generate a 5-legged dog simply by being more specific about the placement of the fifth leg. https://imgur.com/a/jNj98Pc https://imgur.com/a/jNj98Pc Asymmetry is as hard for AI models as it is for evolution to "prompt for" but they're getting better at it.
- DANmode 11mo agoWhat is "a dog"? What is " a dog" to Gemini?
- tarsinge 11mo agoI have only a high level understanding of LLMs but to me it doesn’t seem surprising: they are trying to come up with a textual output of your prompt aggregated to their result that scores high (i.e. is consistent) with their training set. There is no thinking, just scoring consistency. And a dog with 5 legs is so rare or nonexistent in their training set and their resulting weights that it scores so bad they can’t produces an output that accepts it. But how the illusion breaks down in this case is quite funny indeed.
- visioninmyblood 11mo agoI tried this by using an gemini visual agent build with orion from vlm.run. it was able to produce two different images with five leg dog. you need to make it play with itself to improve and correct. https://chat.vlm.run/c/62394973-a869-4a54-a7f5-5f3bb717df5f https://chat.vlm.run/c/62394973-a869-4a54-a7f5-5f3bb717df5f Here is the though process summary(you can see the full thinking the link above): "I have attempted to generate a dog with 5 legs multiple times, verifying each result. Current image generation models have a strong bias towards standard anatomy (4 legs for dogs), making it difficult to consistently produce a specific number of extra limbs despite explicit prompts."
- dostick 11mo agoOf course if you put the leg there it’s confusing. Just give it AGIP logo as a test.