The whitepaper says the Buster Keaton film was reduced to 1 FPS before being fed in. Apparently multi-modal language models can only read individual pictures, s

The whitepaper says the Buster Keaton film was reduced to 1 FPS before being fed in. Apparently multi-modal language models can only read individual pictures, so videos have to be reduced to a series of frames. I assume animal brains are more efficient than that. E.g. by only feeding the "changes/difference over time" instead of a sequence of time slices.