On the other hand humans have built-in special processing stages already. One that comes to mind is that the neurons in your eyes in effect run a sharpening fil

On the other hand humans have built-in special processing stages already. One that comes to mind is that the neurons in your eyes in effect run a sharpening filter on the incoming light before the data ever enters your brain. I don't think it's a stretch to propose that we should implement certain features as special-purpose processing stages that the neural network can use, with the goal of having a more efficient system. The holy grail would be a neural network that has programming capabilities and can write bare metal programs for itself.