I think that one reason that humans are so good at understanding images is that our eyes see video rather than still images. Video lets us see "cause and effect

I think that one reason that humans are so good at understanding images is that our eyes see video rather than still images. Video lets us see "cause and effect" by seeing what happens after something. It also allows us to grasp the 3D structure of things since we will almost always see everything from multiple angles. So long as we just feed a big bunch of stills into training these models, it will struggle to understand how things affect one another.