3 ms·
Ever since OpenAI showed (but did not release) this type of multimodal output with 4o, I have been waiting for this to be available to the general public. It s
by ilaksh 2y ago
Ever since OpenAI showed (but did not release) this type of multimodal output with 4o, I have been waiting for this to be available to the general public.
It seems like really combining visuals at the level of generation capability means language understanding is fully grounded in a richer world model.
I am hoping for a step up in real world common sense intelligence areas like those covered by SimpleBench. Although they are static images, so there might still be room for improvement ad far as physics understanding.
Also, if they can get it to the point of really accurate (probably larger models), this unlocks whole industries in terms of being able to do useful work.
- londons_explore 2y agoIf it can do diagrams, charts, etc, with any kind of accuracy, it would have far more impact. Eg. "I suggest moving the boiler from point A to B on the below map of the factory to reduce piping costs and heat loss"
- ilaksh 2y agoIt's on the right track, but lacks precision. https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%5B%221zOAJhMW5SlgjOo5WrJH-lhxGyxFpYaIm%22%5D,%22action%22:%22open%22,%22userId%22:%22101608810108791339268%22,%22resourceKeys%22:%7B%7D%7D&usp=sharing https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%... Previous try with some interesting introspection: https://drive.google.com/file/d/1SCBbpDo1dAJBAz7bFABk4yBZBuz5j0JD/view?usp=sharing https://drive.google.com/file/d/1SCBbpDo1dAJBAz7bFABk4yBZBuz..., https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%5B%221f30DvcLlDHDyi6waQUmvh8KFY_jgan3I%22%5D,%22action%22:%22open%22,%22userId%22:%22101608810108791339268%22,%22resourceKeys%22:%7B%7D%7D&usp=sharing https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%...