Easiest way I can figure out how to explain my claim. Consider when you use 'partial connectivity', E.G. convolution or pooling layers for local feature extrac

Easiest way I can figure out how to explain my claim. Consider when you use 'partial connectivity', E.G. convolution or pooling layers for local feature extraction on say MNIST. While useful, those partial connection layers are explicitly used because fully connected layers do not have translational invariance. So with a fully connected network, shifting the letter 'i' a few pixels to the right wouldn't match. We choose to discard some of those connections for local feature detection. But as the reason that the fully connected model lacks translational invariance is because it maintains that position data. Note how that is more 'expressive', even if counterproductive for the actual use case. Another lens is the fact that neural networks have extreme simplicity bias. In that they learn only the simplest features to solve a task at hand. If you want to recognize an i, irrespective of the translational location, that bias is useful. But you 'throw away' (in a very loose sense) the positional data to do so. Horses for courses, not good vs bad.