3 ms·

They're proposing a theory of how the brain's cortex must work. Check out some interviews with Jeff Hawkins of Numenta, such as the forementioned interview with

They're proposing a theory of how the brain's cortex must work. Check out some interviews with Jeff Hawkins of Numenta, such as the forementioned interview with Lex Fridman. One essential insight of theirs is that the cortex is physiologically the same everywhere, so any differences between regions would be caused by what happens to be connected to them, rather than something that is inherent to them. This idea of 1000 brains is sort of along those lines.

lostmsu 6y ago
To me this still does not explain what is original or interesting about that idea. Artificial neural networks are "physiologically" the same everywhere too, and have been like that from the inception.
- galaxyLogic 6y ago
  If you train a neural network you get some results. But presumably if you train a 1000 separate neural networks on different measurements made of the same target, and then let those thousand models vote, isn't that a new kind of learning mechanism, compared to having a single neural network that always takes in all the inputs? Further each of the thousand neural nets could have a different learning algorithm.
  - lostmsu 6y ago
    > let those thousand models vote Soon after you start thinking how voting can be implemented, you will quickly realize, that one of the best ways to do it is to connect each of 1000 networks to each of N outputs with some sort of weights (in trivial version - uniform), which effectively turns 1000 networks into 1 network with extra N unit layer, and no weight reuse (e.g. probably a disadvantage) in all but the last layer.
caddemon 6y ago
What is meant by physiologically the same? Because there is a lot of heterogeneity among neurons in the human brain, and there are definitely some macro-level differences between cortical regions that are innate (i.e. happen even with sensory deprivation). So I'd honestly be very surprised if different cortical regions did not have some differences in gene expression patterns that are "hard coded". Which can affect neural dynamics in a subtle way. Determining how much that matters for this hypothesis would require a probably impossible experiment though. FWIW there definitely was an old paper where the animal model (IIRC a ferret?) was deafened and had its visual cortex ablated, and then they wired the lower level visual regions to the auditory cortex, quite early in development. The animal could sort of see, and the auditory cortex had developed a bunch of visual cortex like properties, which is pretty awesome. But at the same time it was certainly not the same thing as using the visual cortex to do so, if you look at the actual behavior the animal had clear deficits in its visual capabilities. Of course that is far from a perfect version of the experiment being described though. Anyway, I read that a long while ago but here is a link to a broader review written by the researcher that did that work, in case anyone is interested in digging further: https://pubmed.ncbi.nlm.nih.gov/16272112/ https://pubmed.ncbi.nlm.nih.gov/16272112/
- deepnotderp 6y ago
  He's referring to the Columnar hypothesis [0] - that all so-called "cortical columns" in the neocortex have the same basic computational behavior. There's controversy about this, the main issue being, as you noted, heterogeneity between columns [1]. [0] https://en.m.wikipedia.org/wiki/Cortical_column https://en.m.wikipedia.org/wiki/Cortical_column [1] https://academic.oup.com/brain/article/125/5/935/328135 https://academic.oup.com/brain/article/125/5/935/328135

lostmsu 6y ago

To me this still does not explain what is original or interesting about that idea. Artificial neural networks are "physiologically" the same everywhere too, and have been like that from the inception.

galaxyLogic 6y ago
If you train a neural network you get some results. But presumably if you train a 1000 separate neural networks on different measurements made of the same target, and then let those thousand models vote, isn't that a new kind of learning mechanism, compared to having a single neural network that always takes in all the inputs? Further each of the thousand neural nets could have a different learning algorithm.
- lostmsu 6y ago
  > let those thousand models vote Soon after you start thinking how voting can be implemented, you will quickly realize, that one of the best ways to do it is to connect each of 1000 networks to each of N outputs with some sort of weights (in trivial version - uniform), which effectively turns 1000 networks into 1 network with extra N unit layer, and no weight reuse (e.g. probably a disadvantage) in all but the last layer.

caddemon 6y ago

What is meant by physiologically the same? Because there is a lot of heterogeneity among neurons in the human brain, and there are definitely some macro-level differences between cortical regions that are innate (i.e. happen even with sensory deprivation). So I'd honestly be very surprised if different cortical regions did not have some differences in gene expression patterns that are "hard coded". Which can affect neural dynamics in a subtle way. Determining how much that matters for this hypothesis would require a probably impossible experiment though. FWIW there definitely was an old paper where the animal model (IIRC a ferret?) was deafened and had its visual cortex ablated, and then they wired the lower level visual regions to the auditory cortex, quite early in development. The animal could sort of see, and the auditory cortex had developed a bunch of visual cortex like properties, which is pretty awesome. But at the same time it was certainly not the same thing as using the visual cortex to do so, if you look at the actual behavior the animal had clear deficits in its visual capabilities. Of course that is far from a perfect version of the experiment being described though. Anyway, I read that a long while ago but here is a link to a broader review written by the researcher that did that work, in case anyone is interested in digging further: https://pubmed.ncbi.nlm.nih.gov/16272112/ https://pubmed.ncbi.nlm.nih.gov/16272112/

deepnotderp 6y ago
He's referring to the Columnar hypothesis [0] - that all so-called "cortical columns" in the neocortex have the same basic computational behavior. There's controversy about this, the main issue being, as you noted, heterogeneity between columns [1]. [0] https://en.m.wikipedia.org/wiki/Cortical_column https://en.m.wikipedia.org/wiki/Cortical_column [1] https://academic.oup.com/brain/article/125/5/935/328135 https://academic.oup.com/brain/article/125/5/935/328135