Here's another thing to consider. The brain is processing "video" in what we might call 3.5 dimensions. (2d images + time with around 6 channels... Plus sound).

Here's another thing to consider. The brain is processing "video" in what we might call 3.5 dimensions. (2d images + time with around 6 channels... Plus sound). Image recognition with a CNN is working on still images, and dropping the time component entirely. RNNs are about handling sequential data, discrete or not... Meanwhile, the hardware in our heads is far more complex and capable than the models we're building. And we're not even sure how that hardware works, or what principles we can abstract out and simplify. A neuron getting electrical and hormonal signals is far more complex than an lstm cell... Does it need to be? Or are there biological requirements built in which we don't understand? And how many evolutionary accidents which never evolved away?