Oh, I should have addressed this part :( In the sentence you quoted, I was referring to my voice recognition example, which is usually trained in supervised fa

Oh, I should have addressed this part :( In the sentence you quoted, I was referring to my voice recognition example, which is usually trained in supervised fashion. Unsupervised pretraining has been thought to help things, but some of the most recent state of the art architectures do not use this step and do just fine. Then there are unsupervised approaches such as the one Google used to find cats. Even in this case, and even when no labels are given, the way you actually train the network is you force it to reconstruct its input patch after passing through the network (The model used is called RICA, which stands for Reconstruction Independent Component Analysis). Then, you are just learning the same function with the same parameters, but the objective now is f^{-1}(f(x)) = x -- so passing the data to the network and propagating it back to input layer should match the original patch well. The reason this works is that the network is an information bottleneck, so the network must find efficient codes to represent the patches with, if it hopes to do a good job in reconstruction. And, as it turns out, little gabor edges are better description of natural patches than individual pixels so this is what it learns. Oh and yes the network finds gabor edges that are just like V1 cells in cortex!!! (Neuroscientists will scold if you tell them this, by the way). What they don't mention is that over the years there were gazillion different models in literature that all give rise to gabor edges as efficient code for images. Simple k-means clustering does this too. Signal processing approaches do, there are many many many ways to arrive at this. Therefore, this piece is more of a sanity check than evidence that we are on the right track to untangling how the visual cortex works (still has basically nothing to do with AI or any kind of cognitive functions).