3 ms·
This is a total guess as I don’t work for OpenAI. But GPT-4 was the first GPT trained using a multimodal dataset of both images and text. Some have speculated t
by uejfiweun 3y ago
This is a total guess as I don’t work for OpenAI. But GPT-4 was the first GPT trained using a multimodal dataset of both images and text. Some have speculated that it’s this additional sensory grounding that allows the model to attain a much more accurate understanding of the universe.
Encourage you to look into Google PALM-E if you want more examples of the astounding capabilities of multimodal models.
- james-revisoai 3y agoThe ScienceQA paper from amazon supports this. In a task where images and text were used for prediction tasks (and often the images showed physical things like pulling a door while the text was a question about physics etc), even the items with only text and no images see higher accuracy.