6 ms·
AI vs. a giraffe with no spots (2023)
- KennyFromIT 2y ago> is there anything unusual about the giraffe? > It appears to be a baby I actually view this as a reasonable response. I don't know anything about giraffes so to me it's entirely plausible that they get their spots later in life.
- jedberg 2y agoI'll be honest, I forgot giraffes have spots. If you had asked me what is abnormal about this giraffe, I would be at a loss. So I guess I'm as dumb as the bot!
- bradleykingz 2y ago[dead]
- dymk 2y agoYou likely wouldn’t if you’d recently seen pictures of giraffes, however
- jedberg 2y agoThat’s the thing! I have recently seen pictures (in kid’s books) and even live ones at the zoo. My brain simply pattern matches the shape not the color. It appears the ML model does the same.
- cameronh90 2y agoI’m the opposite; I don’t think I would have recognised the image as a giraffe at all without the title. Even with the title, my first thought was “that’s a weird deer”.
- cbowal 2y agoAsking ChatGPT with GPT-4o: > please describe this image [image] > The image shows a young giraffe standing on a gravelly surface inside a fenced enclosure. The giraffe has a long neck, characteristic of its species, with a short mane running along the top. Its coat is a light brown color with a paler underbelly. The enclosure has a wire mesh fence and appears to be part of a larger outdoor area, with trees and additional fencing visible in the background. The giraffe is looking directly at the camera with its legs slightly apart, giving it a curious and alert appearance. > anything unusual? > Yes, the giraffe in the image appears unusual because it does not have the typical spotted pattern that giraffes are known for. Instead, its coat is a uniform light brown color. Additionally, the body proportions and the facial features look slightly different from a typical giraffe, suggesting it might be a unique or hybrid species, or possibly an artistically altered image. This giraffe also seems to have a different build, with a more compact body and shorter neck compared to most giraffes.
- josephcsible 2y agoThe point of this post is that AI is horrendously bad at anything not in its training set. Asking an AI that does have it in its training set about it totally misses the point.
- zeroxfe 2y ago> AI is horrendously bad at anything not in its training set. The whole point of AL/ML algorithms is to find generalizations that explicitly apply to data outside the training set. Just because it gets some things wrong doesn't mean that it's bad at anything not in its training set.
- YeGoblynQueenne 2y agoNote date of post above: >> By Janelle Shane On August 28, 2023
- Laremere 2y agoThis article was posted on August 28, 2023. The giraffe was born 31, 2023. So presumably none of the models tested included this particular giraffe in its training data. To directly compare it to GPT-4o, which was released recently, is an invalid comparison. I wouldn't be surprised if GPT-4o does better on novel recognition tasks like this, but you'd need a new novel concept in an image to directly compare GPT-4o and the models tested in this post.
- rossdavidh 2y agoLike, children who are drawing giraffes routinely are able to put spots on them. I am amused at people trying to excuse the AI's here, when they are obviously not only hallucinating a lot of things that aren't in the picture, but also failing to point out something that looking at even one other image of a normal giraffe would allow even the barest amount of intelligence to notice. Much less comparing this image to however many pictures of giraffes are on the internet.
- pheelbert 2y ago[flagged]
- rossdavidh 2y agoThe article is from August, 2023. Of course AI's can do it now; they are trained on lots of articles written about the spotless baby giraffe. The point is whether or not they were able to do it before humans had analyzed it.
- pheelbert 2y agoYou think that the reason it can now is because of some articles that were written about it. From my tests it looks like it can understand the image, its contents, and what's missing. I even asked if it knew about any articles about it (without using web search) and, no, it doesn't. Why do you make baseless assumptions like that? Just try it out instead of speculating.
- aezart 2y agoThe model is not capable of identifying whether it knows a fact or not.
- __loam 2y agoI think the deeper issue is that we didn't know a priori whether these systems could handle this prompt. One way I've heard this described is that the frontier of what computers can do has advanced, but the exact boundary is difficult to know. I don't think one anecdote really makes it clear.
- pheelbert 2y ago[flagged]
- justhere2say 2y ago[flagged]
- pheelbert 2y agoThis was posted less than an hour ago so obviously someone found it of value to be shared. Why would I assume that article was a year old? My assumption of it being recent makes more sense. The fact that it was shared recently makes it worthwhile for me to mention that it's no longer the case with newer, better models.
- 101008 2y ago[flagged]
- Zambyte 2y agoI gave the image to GPT-4o through the Kagi Expert interface. It gives you a description of the image before you ask any questions about it. The initial description did not include anything about the lack of spots, but when I prompted it with the same follow up message from the article "Provide a detailed description of the image and its most distinctive features." it gave me a lot of details, including the lack of spots on the giraffe.
- recursive 2y agoI'm a kagi subscriber, and I haven't heard of the "expert" feature, nor am I able to find it in their AI documentation. Is there a comprehensive list of features somewhere that I'm missing?
- Zambyte 2y agoThe documentation for it is here: https://help.kagi.com/kagi/ai/assistant.html#research https://help.kagi.com/kagi/ai/assistant.html#research I have it through Kagi Ultimate.
- ohmyiv 2y agohttps://help.kagi.com/kagi/ai/assistant.html https://help.kagi.com/kagi/ai/assistant.html It's a version of their beta test of Assistant available to ultimate subscribers.
- not_the_fda 2y agoNow lets amp the game and have AI make military, medical, and financial decisions. And for good measure lets ask it to decide if you are employable, insurable, or suspected criminal.
- GaggiX 2y agoNothingburger, GPT-4o immediately saw that it was a giraffe without spots; the author tried to extrapolate from the performance of two random multimodal models.
- klyrs 2y agoIt's quite likely that this image, and even this article, were included in GPT-4o's training data.
- pdpi 2y agoIt's not a nothingburger. This is a particularly amusing example of one of several known failure modes for this sort of models. Pointing at another model and saying "this didn't fail on that particular input" doesn't change that it's a systemic problem. When the particular failure is the model thinking that "coat" means a piece of clothing, or that it doesn't notice the lack of spots, it's obvious enough a problem that the human operator can detect and work around, but, until we actually solve these issues, it's reckless to the point of negligence to ignore the problem and deploy these tools in unsupervised contexts.
- GaggiX 2y agoInstruct-BLIP and MMICL are not good models, GPT-4o doesn't fail at it no matter what and probably does Gemini and others better models, so yeah it's really a nothingburger, the author tried to extrapolate from the performance of bad models.
- LeoPanthera 2y ago(August 2023)
- rjtjtjjfk 2y ago[flagged]
- deleted 2y ago[deleted]
- kreyenborgi 2y agoHahah, there goes my evening... what a wonderful blog! E.g. https://www.aiweirdness.com/shaped-like-information/ https://www.aiweirdness.com/shaped-like-information/ – the prompt "Please generate a colorful guide to basic geometric shapes, as an aid to children learning to identify basic shapes" gave shape names like: CIRCLE CINCLE SQALE QUARE SQUARE CHALE ʤUARE TLIABLE TRINGLE SVAPE RESNTQON REUAGE RENATLE RECANGE HECTANBIE HOBOZ PSO SEOISUON HEXION FLOTN
- porl 2y agoIf you can't free-hand a FLOTN then you should probably go back to school to learn basic geometry. They are as easy to sketch as HECTANBIEs for crying out loud!
- raincole 2y agoPersonally I'm not sure whether it tells us AI is bad at general concepts or the opposite. If I didn't know spotless giraffe exists, I probably can't tell the creature in this particular image is an adult but spotless giraffe. I might assume they're like birds: the young ones look different from the adults.
- waldarbeiter 2y agoFor everyone interested in what makes a classifier put an image in a certain class I can recommend the excellent paper by Robert Geirhos et al. "ImageNet-trained CNNs are biased towards texture" [1]. The paper is a bit older but it's an excellent read. [1] https://arxiv.org/pdf/1811.12231 https://arxiv.org/pdf/1811.12231
- donkeydonut 2y agoAI's sensitivity constraint - (There is no 'race' or types or skin color, just a giraffe. Leave it at that.. do not pull me into this cess pool.. please)