4 ms·
I’m waiting for someone to make a GPT-style model trained for video and audio prediction (e.g. frame by frame, perhaps) in addition to the existing text predict
by dwaltrip 4y ago
I’m waiting for someone to make a GPT-style model trained for video and audio prediction (e.g. frame by frame, perhaps) in addition to the existing text prediction. Imagine using a significant percentage of YouTube content, for example.
It would probably be insanely expensive. But I feel like it would be almost guaranteed to acquire a world model far richer and more robust than ChatGPT’s.
Human babies learn by watching the world around them. Video frame prediction feels much closer to that than text prediction, and given the wildly impressive results we are seeing with large text prediction models alone, it seems like an obvious next step.
- boredemployee 4y ago>> Human babies learn by watching the world around them. While I understood what you meant, I'd just add that babies learn by a combination of multisensory triggers (so not only _watching_)
- deleted 4y ago[deleted]
- tintor 4y agoThere is VideoGPT paper. It uses small frame sizes, but that will improve with time.
- TomSwirly 4y ago> a significant percentage of YouTube content [...] a world model far richer and more robust than ChatGPT’s There are three objections to this. The first is the astonishingly large amount of CPU power that this would take, given how high bandwidth video is. The second is that it is hard to believe that some thing really coherent could emerge from this, and certainly, it has never been shown. The third is that the world might be "richer" in terms of information-rich but seeing the world through YouTube's eyes would likely be a degrading and incoherent experience.