2 ms·

You may be right that NN activations don't resemble neural receptive fields (though Tomasso Pogio and others disagree [1]), but I strongly suspect that to the e

by joeyo 10y ago

You may be right that NN activations don't resemble neural receptive fields (though Tomasso Pogio and others disagree [1]), but I strongly suspect that to the extent that NNets are able to learn to represent statistical invariants, we will see clear homologs to biological neural networks. The grandparent post made the point that there could intrinsic characteristics to neural computation shared by natural and artificial networks. There's something that feels right about that, but I'd take it a step further: both natural and artificial networks learn to explain the statistics of the natural world in a compact representation. How many possible equivalently compact representations are there? Wouldn't it seem like a monstrous coincidence if there were multiple representations possible yet both DNNs and the mammalian visual system both decided to represent edges first? By the way, you should get your anatomy right if you wish to speak so authoritatively: V1 and V2 are not layers of cortex, but rather adjacent cortical subdivisions with cytoarchitecture that correspond to Brodmann areas 17 and 18. 1. https://mitpress.mit.edu/books/visual-cortex-and-deep-networks https://mitpress.mit.edu/books/visual-cortex-and-deep-networ...

argonaut 10y ago
Notwithstanding the fact that this is just his theory, Tomasso Pogio mostly does not seem to disagree. Reading the summary of that link, it appears he asserts the ventral stream (one of two streams) is well explained by convolutions. So he does not assert the same about the dorsal stream, any other brain region, or any other non-visual domain of deep learning. It bears noting that our visual cortex is one of the simpler areas of our brain, considering that equivalent functionality is in all sorts of animals. It's not a coincidence that DNNs and mammalian visual systems both decide to represent edges first. That's exactly what Yann Lecun was trying to do when he designed convolutional neural nets! You've got the causality mixed up. Yann Lecun explicitly wanted to model the visual system so he came up with a system that would give him the results he wanted (edge detection in the first layer) - which meant linear filters.
- joeyo 10y ago
  Well, the ventral stream is the "what" stream, so that's the natural one to draw comparisons to for the task of representing objects. The "where" pathway for the most part represents space with a place code, that is, different neurons "care about" different parts of space in an increasingly abstract way going from representing eg "the left visual hemifield" early in the dorsal stream to representing "the left side of objects" in parietal cortex. It would be interesting to see if this kind of invariant emerges in NNs. Convolutional networks are able to caption images with labels like "a cake on top of an oven". What do their activations look like? I reject the notion that visual cortex is "simple", but I will concede that it is highly conserved across species. This just means that the representations that it uses are effective. You are no doubt right that Lecun was inspired by the biology of visual cortex (along with theorists before him), but you missed my meaning: can we build a network that doesn't start with edge detectors first that does better? My guess is no.
  - argonaut 10y ago
    This is just speculation, however. (Speculation about captioning neural nets and the direction of evolution and edge detectors)

by joeyo 10y ago

argonaut 10y ago

Notwithstanding the fact that this is just his theory, Tomasso Pogio mostly does not seem to disagree. Reading the summary of that link, it appears he asserts the ventral stream (one of two streams) is well explained by convolutions. So he does not assert the same about the dorsal stream, any other brain region, or any other non-visual domain of deep learning. It bears noting that our visual cortex is one of the simpler areas of our brain, considering that equivalent functionality is in all sorts of animals. It's not a coincidence that DNNs and mammalian visual systems both decide to represent edges first. That's exactly what Yann Lecun was trying to do when he designed convolutional neural nets! You've got the causality mixed up. Yann Lecun explicitly wanted to model the visual system so he came up with a system that would give him the results he wanted (edge detection in the first layer) - which meant linear filters.

joeyo 10y ago
Well, the ventral stream is the "what" stream, so that's the natural one to draw comparisons to for the task of representing objects. The "where" pathway for the most part represents space with a place code, that is, different neurons "care about" different parts of space in an increasingly abstract way going from representing eg "the left visual hemifield" early in the dorsal stream to representing "the left side of objects" in parietal cortex. It would be interesting to see if this kind of invariant emerges in NNs. Convolutional networks are able to caption images with labels like "a cake on top of an oven". What do their activations look like? I reject the notion that visual cortex is "simple", but I will concede that it is highly conserved across species. This just means that the representations that it uses are effective. You are no doubt right that Lecun was inspired by the biology of visual cortex (along with theorists before him), but you missed my meaning: can we build a network that doesn't start with edge detectors first that does better? My guess is no.
- argonaut 10y ago
  This is just speculation, however. (Speculation about captioning neural nets and the direction of evolution and edge detectors)