I've beaten this drum before but another area to look at is network connectivity. When visualizing a neural network we typically see a fully connected network f

I've beaten this drum before but another area to look at is network connectivity. When visualizing a neural network we typically see a fully connected network from one layer to the next, but if you set the value of many weights to zero you'll find that they have absolutely no effect on the function of the network. So basically we're looking at a lot of spurious interactions and this just clouds our thinking. In fact the network topology is what's driving network function here but we're not doing a good job exposing that intuitively. Computational research on Gene Regulatory Networks--which are also modeled with the same NN math--have shown how network topology is really the key driver of function. For anyone who is interested, I published a paper during my PhD that has about 195 citations. It shows that under an evolutionary process that Artificial Gene Regulatory Networks select out spurious network interactions, leaving you with what's functionally necessary (Also see the supplementary). I came from AI before starting my PhD and believe that this could be important for building a foundation for NN as well. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2538912/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2538912/