I'd like to see neuroscience take more of a fundamental role in grounding/situating deep learning approaches. VGG is often mentioned as being roughly analogous

I'd like to see neuroscience take more of a fundamental role in grounding/situating deep learning approaches. VGG is often mentioned as being roughly analogous to the visual cortex, but it differs in important ways. There's all kinds of why questions there. E.g. Why does deep learning work better with (e.g. ReLU) activation based on pooling while biological models use an inhibitory mechanism? Theory on why certain activations work better than others in DL is a little weak imho. Right now ML practitioners just throw a lot of parameter combinations at the wall and see what sticks. That's fine, but it's not really indicative of a robust understanding of model behavior.