In the 1990s, several connectionist and computational neuroscience groups, most notably those led by Jeff Elman and Terry Sejnowski (coincidentally right across

In the 1990s, several connectionist and computational neuroscience groups, most notably those led by Jeff Elman and Terry Sejnowski (coincidentally right across the street from one another at UCSD and The Salk, respectively), studied the self-organization of random NN fields as models of cortical self-organization (called "parcellation"). They discovered that under conditions (vaguely) analogous to mammalian brain development, their highly-interconnected networks self-organized into a structure closely analogous to what we now call Deep Networks: "a self-organizing stack of transducers, well-tuned to their operating environment. A 1995 description stated, "...the infant's brain seems to organize itself under the influence of waves of so-called trophic-factors ... different regions of the brain become connected sequentially, with one layer of tissue maturing before another and so on until the whole brain is mature." (From the wikipedia article on Deep Learning's relation to human brain development).