2 ms·
GPT4 isn’t just doing word prediction though. It has image input in GPT4V, and image output via Dalle. OpenAI are well on the way down the path of multimodality
by zarzavat 3y ago
GPT4 isn’t just doing word prediction though. It has image input in GPT4V, and image output via Dalle. OpenAI are well on the way down the path of multimodality.
It would be fair enough to say that GPT3 doesn’t know what a dog is, it only knows about the word “dog”, but GPT4 has a conception of a dog that is both visual and literary, and as they add more modes the depth of understanding will increase.
- astrange 3y agoDalle is a whole separate model that isn't multimodal; GPT4 just write prompts. If you play with it you can actually see dalle usually doesn't listen to most of what it writes. GPT4V seems to be truly multimodal but is also a different model from GPT4; the writing style is different and it's maybe less intelligent.
- zeusk 3y agohttps://www.adept.ai/blog/fuyu-8b https://www.adept.ai/blog/fuyu-8b Doesn't seem like multimodal inference is as complex as you're making it out to be. It all hinges on what you claim "understanding" to be; are trained weights understanding?