9 ms·
Is It a Duck or a Rabbit? For Google Cloud Vision, Depends on Image Rotation
- WhuzzupDomal 8y agoIt's a seagull not a duck. Don't confuse the dumb AI even more by not knowing what a duck doesn't look like in the first place. Jeez.
- randomsearch 8y agoNo such thing as a seagull, as my ornithologist friend likes to remind me.
- immernerstheime 8y agoplease clarify, genuinely curious, because for me, that Duck is a Seagull, not a freaking duck anywhere near that drawing... Or the one who carries babies, what the freaking ornifkingtologist (bullshit) name for that bird who carries babies??? Have you seen a video of a seagull doing ~pipeline style~ hawaii surfing doing Tubes on the carved waves of the Sea??? If that is not a fucking seagull , my friend , you tell me... what the fck is that??? Haha good one , randomsearch , I upvoted your comment that says: ~no such thing as seagull~ You are right, obviously...you are talking about the analogy of the book The Art of Zen Programming or something like that... the clouds of seagulls that becames a seagull shaped cloud were fucking dissipated by the Master , and no needed to fucking slingshots, no needed to fucking staying sitting at your desk calmly looking at shitt screen like a chariot horse or donkey always look fckng becaude he cannot see the side of the visual field covered by motherfucker stupid(maybe unintentionally enslavers) humans The One who dissipates clouds a Master , stands up from the desk , let any objects on the desk to have his hands completely empty , and goes walking of his cubicle to open air and use his bare hands open like making a number 5 and just easily dissipates the clouds I upvoted randomsearch with 1 point , but you have the free will to give him 10 points, and he has the freewill to give himself 20 or 30 points in his mind... ...freewill always EveryOne thank you
- minimaxir 8y agoCreator of the animation here. Most of the relevant information/context behind the animation (including a link to the repo) is in this Reddit comment: https://reddit.com/r/dataisbeautiful/comments/aydqig/_/ehzyozr/?context=1 https://reddit.com/r/dataisbeautiful/comments/aydqig/_/ehzyo...
- piccogabriele 8y agoWith 3D rotation is even worse
- minimaxir 8y agoTo answer the question why I made the animation: there isn't an ulterior "I found an AI gotcha!" motive, I saw a tweet where the API returned different things depending on orientation and expanded on it. It was also an opportunity to test a few animation hypotheses via gganimate.
- Varcht 8y agoTry this one? https://www.illusionsindex.org/images/illusions/duck-rabbit/tx4.jpg https://www.illusionsindex.org/images/illusions/duck-rabbit/...
- Illniyar 8y agoThat image is a visual illusion. I find it hard myself to detect that it's a rabbit when it's ears are horizontal like a mouth. Not sure what is the purpose of it, is it to show that even computers vision algorithms can get confused by visual illusions?
- 2muchcoffeeman 8y agoI find your response more interesting that the experiment. You don’t just rotate the image in your mind or focus on specific features to bring out the duckiness or rabbit-ness? I can make it more duck or more rabbit as will.
- deleted 8y ago[deleted]
- abhisuri97 8y agoWhen the output switches to rabbit the picture actually resembles a rabbit. I am unsure if this experiment was supposed to be a “haha look how stupid AI is” type thing or not, but it seems like the cloud vision api is performing as intended.
- mikejb 8y agoThe interesting part (IMO) isn't that the AI classifies it as both a rabbit and a duck, but that the classification is dependent on the rotation of the picture. I can somewhat understand how that happens, but I find it's in an interesting observation (rather than a criticism on the system, though the title is somewhat unclear and had me expect something else).
- tanilama 8y agoBut rotation does contain information. 6 and 9 can be considered as a case where rotation SHOULD change the classifier's output. AI does makes dumb predictions from time to time, but I my opinion, this isn't that strong a case. When it rotate upside down, it does look like a rabbit even to me. The more interesting 'failure' here to me, is that while the rotation is smooth, the prediction is not, instead it is flickering, which does raise some interesting questions about what the model's internal distribution surface looks like.
- mikejb 8y ago> AI does makes dumb predictions from time to time, but I my opinion, this isn't that strong a case. I don't think it's meant as an example of a 'dumb prediction'. I think it's an interesting example that inspires (me at least) to think about how this recognition and classification works, and what could cause this effect.
- devin 8y agoThank you for this comment. To me, duck v. rabbit on rotation just proves the system works as intended.
- iscrewyou 8y agoDoes Google Cloud like Duck or Rabbit? That’s where the answer lies. In addition, if Cloud could taste one, it would really help itself with the answer.
- dusted 8y agoLooks like proof to me, that the classification works correctly.
- randomsearch 8y agoIt’s a drawing of a creature that looks a bit like a rabbit or a duck from different angles but is very clearly neither, at best a bad drawing. That’s the failure here - it’s classifying into one of its categories when it shouldn’t be classifying at all.
- mikejb 8y agoIt's one of those optical illusions, where we (humans) do recognize animals. I don't think it's reasonable to limit classifications to the specific case where a drawing perfectly resembles a certain animal. The interesting part (IMO) isn't that the AI classifies it as both a rabbit and a duck, but that the classification is dependent on the rotation of the picture.
- Sharlin 8y agoI don't find it super interesting. The orientation distinction is clearly in the training data, and making the algorithm completely rotation invariant would likely a) be more difficult and b) result in worse classification compared to us humans who very much use orientation as a cue having been evolved in an environment with distinct "up" and "down" directions.
- krehl 8y agoAs the AI has likely not seen anything remotely similar during training, it is quite interesting that it is detecting the animals. As the picture was set up to confuse humans, does it somewhat show, that the representation that the AI learned is similar to the one humans have?
- Sharlin 8y agoThe AI has seen ducks and rabbits. It has build a good enough internal model of "duckness" and "rabbitness" that even more abstract renderings of ducks and rabbits activate the relevant parts of the net. I mean, that's exactly what we want them to do! Figure out the aspects of the input data salient to the classification task and ignore the irrelevant parts. That said, I the response here is likely filtered and normalized to only include "duck" and "rabbit" classifications; after all the bird looks much more like a seagull than a duck.
- MoD411 8y agoit cannot be a rabbit because there is no nose nor mouth
- SmellyGeekBoy 8y agoBut how does it smell?
- deleted 8y ago[deleted]
- zavi 8y agoWAI
- deleted 8y ago[deleted]
- foota 8y agoI wonder whether it would stay consistent if you gave it a solid background line
- Criper1Tookus 8y agoIt would be cool to visualize this as a kind of pie chart, based on where the ears/beak is pointing. Blue for directions where it sees duck, red for rabbit, and empty for neither.
- stared 8y agoWhile the title is clickbaity (as in adversarial examples for fooling neural networks e.g. by adding a baseball ball to a whale to make it shark), I think it shows a nice phenomenon. I.e. a given illusion works similarily for humans and AI alike. Vide "dirty mind" pictures like posting https://images.baklol.com/13_jpegbd9cb76b39e925881bdb2956fd32ac91.jpeg https://images.baklol.com/13_jpegbd9cb76b39e925881bdb2956fd3... to Clarifai https://clarifai.com/models/nsfw-image-recognition-model-e9576d86d2004ed1a38ba0cf39ecb4b1 https://clarifai.com/models/nsfw-image-recognition-model-e95... gives 88% for NSFW.
- wodenokoto 8y agoClickbaity? The title is downright misleading.
- IanCal 8y agoI'm a little confused, it seems quite accurate to me. It's the famous duck/rabbit rotation illusion and the google cloud vision API returns different results depending on rotation. What do you find misleading about it?
- mannykannot 8y agoIt is an example of an adversarial title - one that is sometimes mistaken for clickbait even though it is an accurate summary of the content. This response can be primed by repeated prior training on real clickbait.
- stared 8y agoSee e.g. https://towardsdatascience.com/breaking-neural-networks-with-adversarial-attacks-f4290a9a45aa https://towardsdatascience.com/breaking-neural-networks-with...
- jasode 8y agoI wasn't the one who downvoted you but I agree with IanCal and this blog title didn't trigger my "clickbait" sensor. I disagree that this is an example of "adversarial attack". The famous duck/rabbit illusion has been around since ~1892[1] and therefore was not deliberately constructed to be an "adversary" to image classification neural networks. To me, it's an interesting example of feeding a well-known optical illusion to an AI algorithm and observing its behavior. [1] https://en.wikipedia.org/wiki/Rabbit%E2%80%93duck_illusion https://en.wikipedia.org/wiki/Rabbit%E2%80%93duck_illusion
- EugeneOZ 8y agoOn still (not animated, not rotated) preview I saw rabbit first, then in a second I found it can be a duck also, and now it takes efforts to see rabbit again (but I can do it).
- AlphaWeaver 8y agoI wonder if this was hardcoded/specifically trained to do this for this image?
- usrusr 8y agoThe image was created specifically to fool humans, it's one of the classics in the optical illusion genre. It would be a shame for ML if it required specific trickery on both ends to reach that outcome.
- Gunstig2Snath 8y agoI was ONLY seeing clockwise in all images until the counter-clockwise one went about 8 rotations and all of a sudden I saw it counter-clockwise. Now I can’t unsee it.
- Seldaek 8y agoBut does it see a blue or a white dress?
- ratel 8y agoI quite surprised at the comments on HN so far as nobody seems to see the significance of this. Yes, the image is ambiguous. The point is that Google Cloud Vision gives an unambiguous answer of that image based on the rotation. Transformations of an image are regularly used to improve the results of image recognition. That process fails quit dramatically if in the course of a transformation the answer given is presented with higher confidence than should be.
- tziki 8y agoEh, you use rotations if you're specifically looking for rotation invariance. Also, it's hard to tell how confused the network actually is since it's not really predicting 'probability' of a class (even though the term is commonly used). Quite often neural nets just output the most 'probable' class with an oversized probability estimate due to how the most common classification layer works.
- gambler 8y agoI'm glad that at least someone here sees the problem, but I am not surprised by the typical reaction of AI apologists in this thread. You always get at least one of the two responses: "OMG, this is amazing, it's just like humans. We're probably close to AGI." "Ha-ha, humans are stupid, so the algorithm giving unexpected result is just a proof that it's better and less biased." Here, we have both in response to the same demo. Still, I honestly don't know why some people are so biased in favor of neural nets and have zero interest in edge cases and flaws (the most interesting parts if you want to gain deeper understanding of how the algorithm actually operates). Wishful thinking, I guess.
- cubano 8y agoProbably, and I do hate being the cynic in the room, it's due to the huge salaries and interesting, specific use-case work that NN-based AI is generating these days.
- immernerstheime 8y agoyou are right probably , due to bullshit hiding knowledge from researchers , just for fun , like if there were Gods like greek gods or so , doing it for fun... its not a monopoly of cartelized evil researchers or capistalist owners factorylords, gatekeepers intentionally egoistic possesors of the science knowledge, No , that is not the case Its like it happens naturally, by forces of the natural human stupidity anyway all programming is evil so... dont know even what the fck am I doing commenting in this shitty topic but I have anyway the fckng freewill to fkcng comment... frewill always EveryOne thanks unimportant note the slogan Free Will Always EveryOne is open as in an open beer can and free as in fucking freewill and free beer at the Trio de Carnaval in Brazil carnival and common fucking domain of Nature Itself, myfriend, mor open then wikicommons, its The Fckng Free Domain Open and Public and Common you have the free will to use it freewill always EveryOne thanks But fucking remember: that shit is Not fucking my slogan , I dont have fucking slogans, dont make fucking analogies with fucking me, Dont make analogies with me!!! Fuck! not my fucking slogan! no fucking analogies with me! no analogies! fuck there is no Marco_s fucking Mark left No fucking Mark; no fucking marks; No mark! Cheers! thanks
- ChlorophZek 8y agoWhen I look at the anticlockwise one I can see it as going either direction. When I look at the clockwise one I can only see it going clockwise
- Felz 8y agoIs it concerning that there are short, sudden drops in prediction in the middle of a block otherwise solidly classified as rabbit/duck? I don't know much ML, does anyone know why it'd be so discontinuous?
- minimaxir 8y agoSpecifically, those drops are where the top/bottoms of the image are very slightly cropped out. When making the animation I didn't intend for the occlusion, but the fact that the occlusion causes the prediction to drop to zero is itself an interesting data point. Many objects in real life are occluded.
- Faark 8y agoTo me it just reinforces those algorithms are only comparable to spouting out the intuitive, unconscious first impression of what they see in the image. Like if something passes through your field of vision an you recognize it as lets say a god. But a second later you feel something was off, turn around, look at it in details and determine it to be a weird looking cat? Yeah, those "AI"s so far seem incapable of such deep thinking.
- laichzeit0 8y agoThis is the infamous Duck-Rabit illusion, right? The classifier seems to be doing a good job. https://en.wikipedia.org/wiki/Rabbit%E2%80%93duck_illusion https://en.wikipedia.org/wiki/Rabbit%E2%80%93duck_illusion
- deleted 8y ago[deleted]
- mrashes 8y agoThere is a children's book about this pairing: https://www.amazon.com/Duck-Rabbit-Amy-Krouse-Rosenthal/dp/0811868656 https://www.amazon.com/Duck-Rabbit-Amy-Krouse-Rosenthal/dp/0...
- deleted 8y ago[deleted]
- deleted 8y ago[deleted]
- miguelmota 8y agoThis seems like a serious concern. What's a possible solution to this problem? Should all orientations be considered valid types? so in this case the image should be both a duck and a rabbit as the response?