4 ms·

Don't we still have a lot of video, and other non text real world data to go with? Feels like a possible potential break from there.

• visarga 2y ago
  Generally speaking text only models manage to learn a huge amount about the visual world. So when you put the model train on video it might have less to learn. Video is also less abstract than text, generally. But I am sure we can still extract useful learning from videos, it's probably expensive, but we'll have to do that at some point.