3 ms·

I’m super curious about the data structure that’s being extracted from the image and fed to GPT. I asked it to estimate the size of a figurine based on a ruler

by jcims 3y ago

I’m super curious about the data structure that’s being extracted from the image and fed to GPT. I asked it to estimate the size of a figurine based on a ruler in the image, and it nailed it. https://imgur.com/a/gsiID4v https://imgur.com/a/gsiID4v At first I just thought it was able taking an array of objects identified in the photo with bounding box coordinates and then just figuring out what the answer is. But then i asked it to describe Worf’s hands in this image and it did a great job. https://imgur.com/a/orpDszj https://imgur.com/a/orpDszj I’m super curious. I’ve done some basic attempts to get it to spit out the actual raw data but it won’t. I’m sure folks that are good at that could.

simonw 3y ago
I don't think that's the right way to think about it. It's not running an OCR-style algorithm to export data and then passing that data to the LLM for further processing. Instead, the model itself has been trained on both text and images at the same time. So the image features end up as a weird ball of floating point numbers mixed together with the floating point numbers for the textual representation of the words in the prompt. I'd love to understand this more. Strings of text get tokenized into a sequence of integer tokens - is there a similar tokenization step for image inputs? What does that look like?
- jcims 3y ago
  Ah! Do you think this is like an adjunct or fine-tuning kind of thing or do they retrain GPT 4+ images? That seems like a ginormous task.
  - simonw 3y ago
    When they launched GPT-4 back in March they talked about its image capabilities - I think they trained it into the model from the start, but then spent several extra months on additional safety research, as described in their paper: https://cdn.openai.com/papers/GPTV_System_Card.pdf https://cdn.openai.com/papers/GPTV_System_Card.pdf
    - GaggiX 3y ago
      I don't think they have trained the model from scratch on images too, but more like Flamingo or similar methods, and the LLM model is at most finetuned but not trained from scratch. If the model was trained from scratch then I would expect it to be able to develop new emergent behavior like being much better at ascii art, SVG, OpenSCAD scripting at least in the case the model was trained on the usual masked objective but not only on text tokens but image tokens too.
- mk_stjames 3y ago
  The entire GPT4 model was not retrained to utilize images, instead I think it is much more like show in [1] where an extra layer(s) are learned that take another pretrained image model and map that model to the latent space of the LLM. In [1] this was just a single linear layer. I found it pretty shocking how simple it was. I believe OpenAI is doing something similar. [1] https://arxiv.org/pdf/2304.10592.pdf https://arxiv.org/pdf/2304.10592.pdf
  - potatoman22 3y ago
    GPT4 was trained to be multimodal from the start.
- jug 3y ago
  I wonder if OpenAI themselves even know how the resulting black box actually operates? They know how to train it, sure, but do they understand how it approaches the end results?
  - simonw 3y ago
    I'm pretty confident they don't, and I think they might openly admit that. There's a lot of ongoing research into interpretability of these models. Anthropic put out a major new paper about that a few days ago: https://www.anthropic.com/index/decomposing-language-models-into-understandable-components https://www.anthropic.com/index/decomposing-language-models-...

by jcims 3y ago

simonw 3y ago

I don't think that's the right way to think about it. It's not running an OCR-style algorithm to export data and then passing that data to the LLM for further processing. Instead, the model itself has been trained on both text and images at the same time. So the image features end up as a weird ball of floating point numbers mixed together with the floating point numbers for the textual representation of the words in the prompt. I'd love to understand this more. Strings of text get tokenized into a sequence of integer tokens - is there a similar tokenization step for image inputs? What does that look like?

jcims 3y ago
Ah! Do you think this is like an adjunct or fine-tuning kind of thing or do they retrain GPT 4+ images? That seems like a ginormous task.
- simonw 3y ago
  When they launched GPT-4 back in March they talked about its image capabilities - I think they trained it into the model from the start, but then spent several extra months on additional safety research, as described in their paper: https://cdn.openai.com/papers/GPTV_System_Card.pdf https://cdn.openai.com/papers/GPTV_System_Card.pdf
  - GaggiX 3y ago
    I don't think they have trained the model from scratch on images too, but more like Flamingo or similar methods, and the LLM model is at most finetuned but not trained from scratch. If the model was trained from scratch then I would expect it to be able to develop new emergent behavior like being much better at ascii art, SVG, OpenSCAD scripting at least in the case the model was trained on the usual masked objective but not only on text tokens but image tokens too.
mk_stjames 3y ago
The entire GPT4 model was not retrained to utilize images, instead I think it is much more like show in [1] where an extra layer(s) are learned that take another pretrained image model and map that model to the latent space of the LLM. In [1] this was just a single linear layer. I found it pretty shocking how simple it was. I believe OpenAI is doing something similar. [1] https://arxiv.org/pdf/2304.10592.pdf https://arxiv.org/pdf/2304.10592.pdf
- potatoman22 3y ago
  GPT4 was trained to be multimodal from the start.
jug 3y ago
I wonder if OpenAI themselves even know how the resulting black box actually operates? They know how to train it, sure, but do they understand how it approaches the end results?
- simonw 3y ago
  I'm pretty confident they don't, and I think they might openly admit that. There's a lot of ongoing research into interpretability of these models. Anthropic put out a major new paper about that a few days ago: https://www.anthropic.com/index/decomposing-language-models-into-understandable-components https://www.anthropic.com/index/decomposing-language-models-...