3 ms·
[tangential to this really cool demo] JPEG images being the only possible interface to GPT-4 feels wasteful. the human eye works the delta between "frames", not
by adtac 3y ago
[tangential to this really cool demo] JPEG images being the only possible interface to GPT-4 feels wasteful. the human eye works the delta between "frames", not the image itself. I wonder if the next big step that would allow real-time video processing at high resolutions is to have the model's internal state operate on keyframes and deltas similar to how video codecs like MPEG work.
- zwily 3y agoWhen Google talks about Gemini's "multi-modal", they include "video" in the list of modes. It's totally possible they don't actually mean video, and just mean frames like in this demo. They haven't elaborated on it anywhere that I've seen.
- dannyw 3y agoTheir technical report clarifies that video is just a sequence of frames fed as images.