3 ms·

FWIW I tried the first couple of examples in ChatGPT 4o and couldn't replicate this. For example: "The animal in the image is a chicken, and it appears to have

by runako 1y ago

FWIW I tried the first couple of examples in ChatGPT 4o and couldn't replicate this. For example: "The animal in the image is a chicken, and it appears to have four legs. However, chickens normally have only two legs. The presence of four legs suggests that the image may have been digitally altered or artificially generated." I don't have a good explanation for why I got different results.

roywiggins 1y ago
I gave ChatGPT some miswritten Braille a while ago and it completely, but confidently, messed it up. The sign reads "no smoking" but the braille doesn't. ChatGPT 1) read the English lettering first and then hallucinated the braille and the 2) when given only the braille, failed almost as hard. It even generated fake transcriptions in Unicode braille characters. https://chatgpt.com/share/683f3e7d-0dfc-8005-b6c9-99e3d39ff424 https://chatgpt.com/share/683f3e7d-0dfc-8005-b6c9-99e3d39ff4... https://chatgpt.com/share/683f3e49-9c58-8005-99a6-c3a919838b3d https://chatgpt.com/share/683f3e49-9c58-8005-99a6-c3a919838b...
- Workaccount2 1y ago
  This is hard to understand without the original images, it looks like OpenAI doesn't serve them in the share link.
  - roywiggins 1y ago
    Annoying. The actual braille on the sign was "⠁⠒⠑⠎⠎⠊⠼" which I gather means "accessible" in abbreviated braille. None of my attempts got it to even transcribe it to Unicode characters properly. I got "elevator", "friend", etc. Just wildly making stuff up and completely useless, even when it wasn't distracted by the No Smoking sign (in the second case I cropped out the rest of the sign). And in all cases, supremely confident. This seems like something a VLM should handle very easily, but instead I got pure nonsense. https://www.facebook.com/share/p/12Gw55Gr2SZ/ https://www.facebook.com/share/p/12Gw55Gr2SZ/
    - dragonwriter 1y ago
      > This seems like something a VLM should handle very easily Not if its training data doesn't include braille as first class but has lots of braille signage with bad description (e.g., because people assumed the accompanying English matches the braille.) This could very well be the kind of mundane AI bias problem that the x-risk and tell-me-how-to-make-WMD concerns have shifted concerns about problems in AI away from.
      - roywiggins 1y ago
        I'd wager that correctly labeled braille far exceeds dumb braille, and when presented with just the braille it flat out hallucinated braille characters that weren't there. It didn't seem to actually be parsing the dots at all. My theory is that it has hardly seen any braille, despite it insisting that it knows how to read it.
inerte 1y ago
I took a screenshot of the chicken, so low res, and got {4} https://chatgpt.com/share/683f4506-ae18-800f-8c27-5c5e91429adb https://chatgpt.com/share/683f4506-ae18-800f-8c27-5c5e91429a... Also I think the authors used the API, and maybe there are differences between the API and chatgpt.com behavior...
- runako 1y ago
  I could rant for quite a while about how OpenAI and Anthropic manage their apps vs their APIs. It's really quite strange that they both landed on the solution of non-public APIs that perform differently than their public APIs.
- simonw 1y ago
  ChatGPT is running a special model but it's also available through the API: https://platform.openai.com/docs/models/chatgpt-4o-latest https://platform.openai.com/docs/models/chatgpt-4o-latest The system prompt may still make a difference though.
- anguyen8 1y ago
  https://imgur.com/cO7eFNt https://imgur.com/cO7eFNt o3 Chat is also similarly wrong, saying {4}.
vokhanhan25 1y ago
You should try with other models besides GPT-4o, because in the paper they also show that GPT4.1 (~GPT-4o) gives 4 legs instead of 2 legs.
- runako 1y ago
  I mean perhaps! But that would undermine the conclusion of the article.
dwringer 1y ago
Speculating, I would imagine that different prompts submitted along with the image might elicit wildly different behavior in how a multi modal VLM may respond to a given image, potentially affecting the relative tendency to upweight its effective inferences from prior training versus focusing more primarily on the new image itself.
michaelt 1y ago
> FWIW I tried the first couple of examples in ChatGPT 4o and couldn't replicate this. I can replicate the flag examples from Figure 15 in the paper, if not the Adidas one from Figure 9: https://chatgpt.com/share/683f7c3a-b318-8011-9759-c495db2556b7 https://chatgpt.com/share/683f7c3a-b318-8011-9759-c495db2556... it even confirms its wrong answer when asked to check again.
obscurette 1y ago
I suspect that responses are altered/corrected based on what people query from popular online models. I have had several occasions that I ask some "How do I ... in X software?" question some day and model keeps hallucinating nonexistant config options regardless how many times I keep saying "This option doesn't exist in software X". But if I asked the same question some days later, the answer was completely different and made even some sense.
deleted 1y ago
[deleted]

by runako 1y ago

roywiggins 1y ago

I gave ChatGPT some miswritten Braille a while ago and it completely, but confidently, messed it up. The sign reads "no smoking" but the braille doesn't. ChatGPT 1) read the English lettering first and then hallucinated the braille and the 2) when given only the braille, failed almost as hard. It even generated fake transcriptions in Unicode braille characters. https://chatgpt.com/share/683f3e7d-0dfc-8005-b6c9-99e3d39ff424 https://chatgpt.com/share/683f3e7d-0dfc-8005-b6c9-99e3d39ff4... https://chatgpt.com/share/683f3e49-9c58-8005-99a6-c3a919838b3d https://chatgpt.com/share/683f3e49-9c58-8005-99a6-c3a919838b...

Workaccount2 1y ago
This is hard to understand without the original images, it looks like OpenAI doesn't serve them in the share link.
- roywiggins 1y ago
  Annoying. The actual braille on the sign was "⠁⠒⠑⠎⠎⠊⠼" which I gather means "accessible" in abbreviated braille. None of my attempts got it to even transcribe it to Unicode characters properly. I got "elevator", "friend", etc. Just wildly making stuff up and completely useless, even when it wasn't distracted by the No Smoking sign (in the second case I cropped out the rest of the sign). And in all cases, supremely confident. This seems like something a VLM should handle very easily, but instead I got pure nonsense. https://www.facebook.com/share/p/12Gw55Gr2SZ/ https://www.facebook.com/share/p/12Gw55Gr2SZ/
  - dragonwriter 1y ago
    > This seems like something a VLM should handle very easily Not if its training data doesn't include braille as first class but has lots of braille signage with bad description (e.g., because people assumed the accompanying English matches the braille.) This could very well be the kind of mundane AI bias problem that the x-risk and tell-me-how-to-make-WMD concerns have shifted concerns about problems in AI away from.
    - roywiggins 1y ago
      I'd wager that correctly labeled braille far exceeds dumb braille, and when presented with just the braille it flat out hallucinated braille characters that weren't there. It didn't seem to actually be parsing the dots at all. My theory is that it has hardly seen any braille, despite it insisting that it knows how to read it.

inerte 1y ago

I took a screenshot of the chicken, so low res, and got {4} https://chatgpt.com/share/683f4506-ae18-800f-8c27-5c5e91429adb https://chatgpt.com/share/683f4506-ae18-800f-8c27-5c5e91429a... Also I think the authors used the API, and maybe there are differences between the API and chatgpt.com behavior...

runako 1y ago
I could rant for quite a while about how OpenAI and Anthropic manage their apps vs their APIs. It's really quite strange that they both landed on the solution of non-public APIs that perform differently than their public APIs.
simonw 1y ago
ChatGPT is running a special model but it's also available through the API: https://platform.openai.com/docs/models/chatgpt-4o-latest https://platform.openai.com/docs/models/chatgpt-4o-latest The system prompt may still make a difference though.
anguyen8 1y ago
https://imgur.com/cO7eFNt https://imgur.com/cO7eFNt o3 Chat is also similarly wrong, saying {4}.

vokhanhan25 1y ago

You should try with other models besides GPT-4o, because in the paper they also show that GPT4.1 (~GPT-4o) gives 4 legs instead of 2 legs.

runako 1y ago
I mean perhaps! But that would undermine the conclusion of the article.

dwringer 1y ago

Speculating, I would imagine that different prompts submitted along with the image might elicit wildly different behavior in how a multi modal VLM may respond to a given image, potentially affecting the relative tendency to upweight its effective inferences from prior training versus focusing more primarily on the new image itself.

michaelt 1y ago

> FWIW I tried the first couple of examples in ChatGPT 4o and couldn't replicate this. I can replicate the flag examples from Figure 15 in the paper, if not the Adidas one from Figure 9: https://chatgpt.com/share/683f7c3a-b318-8011-9759-c495db2556b7 https://chatgpt.com/share/683f7c3a-b318-8011-9759-c495db2556... it even confirms its wrong answer when asked to check again.

obscurette 1y ago

I suspect that responses are altered/corrected based on what people query from popular online models. I have had several occasions that I ask some "How do I ... in X software?" question some day and model keeps hallucinating nonexistant config options regardless how many times I keep saying "This option doesn't exist in software X". But if I asked the same question some days later, the answer was completely different and made even some sense.

deleted 1y ago

[deleted]