3 ms·

Again this is just a hypothesis and nothing has come from it despite the fact that it is very old. There isn't any good reason for it to be true and no one has

Again this is just a hypothesis and nothing has come from it despite the fact that it is very old. There isn't any good reason for it to be true and no one has done anything to show that it is.

HarHarVeryFunny 3y ago
Not sure what you're referring to - what's a hypothesis/old ?
- foobarqux 3y ago
  Pretty much any claim that the brain (outside the visual cortex) work anything like the way neural nets do.
  - HarHarVeryFunny 3y ago
    So why do you believe that the visual cortex works similarly to an ANN, but the rest of the cortex doesn't ?! Of course we know that CNNs and the visual cortex both learn similar low level orientated-line feature detectors, etc, so there is evidence of them operating in similar fashion. The thing is, the entire cortex is a very regular structure with the same 6-layer architecture across all areas and same thalamo-cortical loop connectivity. So, it's very unlikely that one area of cortex is operating in a fashion much different to any other area. If you're willing to accept that the visual cortex has close parallels to a CNN in terms of how it is operating, then that is highly suggestive that the abstraction of a CNN/ANN does indeed capture the essence of at least some aspects of cortical operation. Nobody is saying that the brain and ANNs are exactly the same, but there seems to be close enough correspondence to at least some types of ANN architecture that lessons learned there can inspire understanding of the latter. Another example - before seq-2-seq neural net language translation, would you have believed that an a variable length sentence could have it's meaning captured by a fixed size sentence embedding ? Intuition might have suggested that some complex structured variable length representation would be needed, but now we know that's not the case, and it helps understand how similar representations are likely also used in the brain.
    - foobarqux 3y ago
      > So why do you believe that the visual cortex works similarly to an ANN, but the rest of the cortex doesn't ?! Because there are actual (unethical on humans) experiments on cats (which have similar visual systems to humans) so we know some things about how the visual system works. There is nothing equivalent for the rest of the mind. > Nobody is saying that the brain and ANNs are exactly the same, but there seems to be close enough correspondence to at least some types of ANN architecture that lessons learned there can inspire understanding of the latter. Again this is a hypothesis, the burden is on the person making that claim to provide evidence. > Another example - [...] This is a much worse example because we specifically know that LLMs don't work the way the human language faculty do because they can "learn" languages that are impossible for humans.
      - HarHarVeryFunny 3y ago
        You completely ignored the point about the uniform nature of the cortex. It doesn't make sense to say say "well, ok, maybe this bit works like an ANN, but the rest works differently". > we specifically know that LLMs don't work the way the human language faculty do No we don't, because we still don't know how human language works. People are still debating Chompsky innate-ism. Again, LLM processing of language offers interesting clues - not just embeddings, but also the fact that language can be learnt with a very simple architecture (transformer) with zero support for "universal grammar" or any such priors. You seem to be missing the spirit of this whole conversation and the original linked article - it's not about claiming/hypothesizing that the brain works the same as any and all ANN architectures or principles, but rather that one can draw inspiration from ANNs to develop theories and intuitions (which still need to be tested) for how the brain may be working.
        foobarqux 3y ago
        > You completely ignored the point about the uniform nature of the cortex. It doesn't make sense to say say "well, ok, maybe this bit works like an ANN, but the rest works differently". The "uniform nature of the cortex" implies far too much. We know that dramatically different processes happen in different parts of the brain. That needs to be explained and it's where all the interesting questions are. You might as well say every part of the body is made of cells so they all work the same. That might be true at some level of abstraction but it's completely useless. > No we don't, because we still don't know how human language works. We do know some ways they don't work: namely neural nets because it can be shown that neural nets can learn languages that human cannot. > People are still debating Chompsky innate-ism. Again, LLM processing of language offers interesting clues - not just embeddings, but also the fact that language can be learnt with a very simple architecture (transformer) with zero support for "universal grammar" or any such priors. This tells you nothing about the language faculty just as building a neural net to predict the weather tells you nothing about how the weather operates. > You seem to be missing the spirit of this whole conversation and the original linked article - it's not about claiming/hypothesizing that the brain works the same as any and all ANN architectures or principles, but rather that one can draw inspiration from ANNs to develop theories and intuitions (which still need to be tested) for how the brain may be working. People have been making claims that connectionist models will lead to some insight for decades and there isn't much to show for it. No one cares where someone derives their inspiration for their hypotheses, come back when you have some result to show.

Again this is just a hypothesis and nothing has come from it despite the fact that it is very old. There isn't any good reason for it to be true and no one has done anything to show that it is.

HarHarVeryFunny 3y ago

Not sure what you're referring to - what's a hypothesis/old ?

foobarqux 3y ago
Pretty much any claim that the brain (outside the visual cortex) work anything like the way neural nets do.
- HarHarVeryFunny 3y ago
  So why do you believe that the visual cortex works similarly to an ANN, but the rest of the cortex doesn't ?! Of course we know that CNNs and the visual cortex both learn similar low level orientated-line feature detectors, etc, so there is evidence of them operating in similar fashion. The thing is, the entire cortex is a very regular structure with the same 6-layer architecture across all areas and same thalamo-cortical loop connectivity. So, it's very unlikely that one area of cortex is operating in a fashion much different to any other area. If you're willing to accept that the visual cortex has close parallels to a CNN in terms of how it is operating, then that is highly suggestive that the abstraction of a CNN/ANN does indeed capture the essence of at least some aspects of cortical operation. Nobody is saying that the brain and ANNs are exactly the same, but there seems to be close enough correspondence to at least some types of ANN architecture that lessons learned there can inspire understanding of the latter. Another example - before seq-2-seq neural net language translation, would you have believed that an a variable length sentence could have it's meaning captured by a fixed size sentence embedding ? Intuition might have suggested that some complex structured variable length representation would be needed, but now we know that's not the case, and it helps understand how similar representations are likely also used in the brain.
  - foobarqux 3y ago
    > So why do you believe that the visual cortex works similarly to an ANN, but the rest of the cortex doesn't ?! Because there are actual (unethical on humans) experiments on cats (which have similar visual systems to humans) so we know some things about how the visual system works. There is nothing equivalent for the rest of the mind. > Nobody is saying that the brain and ANNs are exactly the same, but there seems to be close enough correspondence to at least some types of ANN architecture that lessons learned there can inspire understanding of the latter. Again this is a hypothesis, the burden is on the person making that claim to provide evidence. > Another example - [...] This is a much worse example because we specifically know that LLMs don't work the way the human language faculty do because they can "learn" languages that are impossible for humans.
    - HarHarVeryFunny 3y ago
      You completely ignored the point about the uniform nature of the cortex. It doesn't make sense to say say "well, ok, maybe this bit works like an ANN, but the rest works differently". > we specifically know that LLMs don't work the way the human language faculty do No we don't, because we still don't know how human language works. People are still debating Chompsky innate-ism. Again, LLM processing of language offers interesting clues - not just embeddings, but also the fact that language can be learnt with a very simple architecture (transformer) with zero support for "universal grammar" or any such priors. You seem to be missing the spirit of this whole conversation and the original linked article - it's not about claiming/hypothesizing that the brain works the same as any and all ANN architectures or principles, but rather that one can draw inspiration from ANNs to develop theories and intuitions (which still need to be tested) for how the brain may be working.
      - foobarqux 3y ago
        > You completely ignored the point about the uniform nature of the cortex. It doesn't make sense to say say "well, ok, maybe this bit works like an ANN, but the rest works differently". The "uniform nature of the cortex" implies far too much. We know that dramatically different processes happen in different parts of the brain. That needs to be explained and it's where all the interesting questions are. You might as well say every part of the body is made of cells so they all work the same. That might be true at some level of abstraction but it's completely useless. > No we don't, because we still don't know how human language works. We do know some ways they don't work: namely neural nets because it can be shown that neural nets can learn languages that human cannot. > People are still debating Chompsky innate-ism. Again, LLM processing of language offers interesting clues - not just embeddings, but also the fact that language can be learnt with a very simple architecture (transformer) with zero support for "universal grammar" or any such priors. This tells you nothing about the language faculty just as building a neural net to predict the weather tells you nothing about how the weather operates. > You seem to be missing the spirit of this whole conversation and the original linked article - it's not about claiming/hypothesizing that the brain works the same as any and all ANN architectures or principles, but rather that one can draw inspiration from ANNs to develop theories and intuitions (which still need to be tested) for how the brain may be working. People have been making claims that connectionist models will lead to some insight for decades and there isn't much to show for it. No one cares where someone derives their inspiration for their hypotheses, come back when you have some result to show.