Having read some very introductory stuff about human visual system, I'd be willing to guess that a partial reason is that the human vision works on the input le

Having read some very introductory stuff about human visual system, I'd be willing to guess that a partial reason is that the human vision works on the input level quite differently than "observe NxM colored pixels in grid". The act of constructing what we'd think as a single "view" of our surroundings (that one intuitively might equate with a still pixel image) includes lots of low-level processing. (E.g. Everyone probably has heard about the blind spot at the point where optic nerve 'connects' to retina. But I was surprised to learn that most of the stuff in our FOV is seen very poorly, and only the tiny area called macula is responsible for the high acuity vision. And the macula is capable of covering only a small spot in our FOV at one time: what we'd think as a single view is really constructed in a post-processing step as our eyes move and scan our surroundings. My amateur-level explanation might be a bit off, but the point is, it stands to reason that kind of "dynamic" processing system is more robust (especially to small pixel level "attacks" as in the linked paper) than simply constructing mappings of still pixel grids to outputs, as currently done by CNNs.)