5 ms·

This post broke my brain a little as it goes totally against my mental model of how an LLM is processing images. That is, I don't know why it is processing the

This post broke my brain a little as it goes totally against my mental model of how an LLM is processing images. That is, I don't know why it is processing the content of the image, and then running that through something that then "executes" the text of the image. So it has some pipeline like -> LLM/OCR Image to Text -> Text to Command Processing. Why is the text description of the image being processed at all and not simply just being output? As far as I know, we've had engines for detecting sentiment, objects, etc from images for a long time (way before ChatGPT), and they didn't have issues where they would execute a URL found in an image, they would just output the characters... Can someone who understands this better explain wth is going on here?

Michelangelo11 3y ago
Interesting that you say that, because this lines up exactly with my mental models of LLM image processing and LLM functioning in general. One of the defining features of LLMs is that, unlike in regular programming, there's no separation between data and instructions -- they're one and the same -- and that's what we're seeing here. Also, LLMs tend to focus more heavily on the instructions they see last (because of limitations in the attention mechanism, I think? But IIRC nobody has come up with a full explanation yet), and that's what we're seeing here too.
- saberience 3y ago
  So the original release of ChatGPT would call a URL you entered in the chat? I don't recall it acting that way. In the linked post ChatGPT literally goes to a URL from an image... i.e. it calls something like curl with that URL. This definitely seems like something adjacent to an LLM.
  - simonw 3y ago
    ChatGPT didn't grow the ability to visit URLs until they introduced ChatGPT Browse - though it frequently hallucinated that it could: https://simonwillison.net/2023/Mar/10/chatgpt-internet-access/ https://simonwillison.net/2023/Mar/10/chatgpt-internet-acces... The exfiltration demo in the blog post doesn't show ChatGPT fetching a URL: it shows it outputting an image tag with an src= pointing at an external URL, which the user's browser then attempts to load.
  - Michelangelo11 3y ago
    Well, the original release had browsing disabled, so it couldn't access any URL.
simonw 3y ago
See my comment here: https://news.ycombinator.com/item?id=37880515 https://news.ycombinator.com/item?id=37880515 I don't think it's accurate to think of the image processing as a separate pipeline here, similar to OCR on object detection. My understanding is that the image is treated the same as the text - it's all input into the weird giant matrices of numbers that makes up the model. GPT4-V isn't GPT4 plus tools for analyzing images: it's GPT4 with image processing built right into the model itself. Is it even right to call it a "Large Language Model" at this point? I'm not sure.
- Michelangelo11 3y ago
  Large Comprehension Model :)

Michelangelo11 3y ago

Interesting that you say that, because this lines up exactly with my mental models of LLM image processing and LLM functioning in general. One of the defining features of LLMs is that, unlike in regular programming, there's no separation between data and instructions -- they're one and the same -- and that's what we're seeing here. Also, LLMs tend to focus more heavily on the instructions they see last (because of limitations in the attention mechanism, I think? But IIRC nobody has come up with a full explanation yet), and that's what we're seeing here too.

saberience 3y ago
So the original release of ChatGPT would call a URL you entered in the chat? I don't recall it acting that way. In the linked post ChatGPT literally goes to a URL from an image... i.e. it calls something like curl with that URL. This definitely seems like something adjacent to an LLM.
- simonw 3y ago
  ChatGPT didn't grow the ability to visit URLs until they introduced ChatGPT Browse - though it frequently hallucinated that it could: https://simonwillison.net/2023/Mar/10/chatgpt-internet-access/ https://simonwillison.net/2023/Mar/10/chatgpt-internet-acces... The exfiltration demo in the blog post doesn't show ChatGPT fetching a URL: it shows it outputting an image tag with an src= pointing at an external URL, which the user's browser then attempts to load.
- Michelangelo11 3y ago
  Well, the original release had browsing disabled, so it couldn't access any URL.

simonw 3y ago

See my comment here: https://news.ycombinator.com/item?id=37880515 https://news.ycombinator.com/item?id=37880515 I don't think it's accurate to think of the image processing as a separate pipeline here, similar to OCR on object detection. My understanding is that the image is treated the same as the text - it's all input into the weird giant matrices of numbers that makes up the model. GPT4-V isn't GPT4 plus tools for analyzing images: it's GPT4 with image processing built right into the model itself. Is it even right to call it a "Large Language Model" at this point? I'm not sure.

Michelangelo11 3y ago
Large Comprehension Model :)