3 ms·

I think the Economist article is exactly right - that despite the massive differences between ANNs and the brain, ANNs are indeed highly suggestive of how some

I think the Economist article is exactly right - that despite the massive differences between ANNs and the brain, ANNs are indeed highly suggestive of how some aspects of the brain appear to work. People can criticize the shortcomings of GPT-4, but it's hard to argue that it's at least capable of some level of reasoning (or functionally equivalent if you object to that word!). It's not yet clear exactly how a Transformer works other than at mechanical level of the model architecture (vs the LLM "running on" the architecture), but we are at least starting to glean some knowledge of how the trained model is operating... It seems that pairs of attention heads in consecutive layers are acting in coordination as "induction heads" that in one case are performing a kind of analogical(?) A'B' => AB match-and-copy type of operation. The induction head causes a context token A to be matched (via "attention" key query) with an earlier token A' whose following token B' then causes related token B to be copied to the residual stream at position following A. This seems a very basic type of operation, and no doubt there's a lot more interpretability research to be done, but given the resulting reasoning/cognitive power (even in absense of any working memory or looping!), it seems we don't need to go looking for overly complex exotic mechanisms to begin to understand how the cortex may be operating. It's easy to imagine how this same type of embedded key matching might work in the cortex, perhaps with cortical columns acting as complex pattern matchers. Perhaps the brain's well known ~7 item working memory corresponds to a "context" of sorts that is updated in same way as induction heads update the residual stream. Anything I've written here about correspondence between transformer and cortex is of course massive speculation, but the point is that the ANN's operation does indeed start to suggest how the brain, operating on similar sparse/embedded representations, may be working.

amelius 3y ago
But (how) does the human brain perform backpropagation?
- HarHarVeryFunny 3y ago
  It probably doesn't use backpropagation of gradients. Instead, the cortex appears to be a prediction engine that uses error feedback (perceptual reality vs prediction) to minimize prediction errors in a conceptually similar type of way. If every "layer" (cortical patch) is doing it's own prediction and receiving feedback, then you don't need any error propagation from one layer to the next.
  - marcosdumay 3y ago
    You just can't go changing the layers of a neural network independently of each other if you are doing guided optimization. For a start, it's not even a given if increasing some weight will increase or decrease the result. The neurons are all tightly coupled. You can do it on unguided optimization, what is one of the reasons I strongly suspect our brains use something similar to simulated annealing.
    - HarHarVeryFunny 3y ago
      Sure, we don't know the exact details (Geoff Hinton spent much of his career trying to answer this question), but at the big picture level it does seem clear that the cortex is a prediction engine that minimizes prediction errors by feedback, and most likely does so in a localized way. Exactly how these prediction updates work is unknown. Could you expand a bit on how you think simulated annealing could work?
      - deleted 3y ago
        [deleted]
      - marcosdumay 3y ago
        Hum... I have no model for what actually happens. It's just that simulated annealing appears quite often on the biologic world, we never found any coordination communication channel that could create something more "intelligent" (AFAIK, there are just a few "neighborhood to global" analog signals available), and there isn't a multiplicity of attempts interacting with each other (again, there is no channel for that) that would lead to a parallel evolutionary strategy. Also, when we learn a mechanical task, our errors tend to cool down around our successes, instead of following a direction. But again, I have no idea of any mechanism that is there. It's more because all the others seem to have been eliminated.
- canjobear 3y ago
  Approximately, via predictive coding https://arxiv.org/abs/2006.04182 https://arxiv.org/abs/2006.04182
foobarqux 3y ago
This is just false. Outside of the visual cortex there isn't any evidence that brains work anything like GPT-4 or neural nets. Producing the same outputs isn't evidence of anything. At best you have just stated a hypothesis of how the brain might work, not any actual evidence supporting it.
- HarHarVeryFunny 3y ago
  Sure - you can't just look at how an ANN works and assume that's how the brain does that too, but the ANN's operation can act as inspiration to suggest or confirm the way the brain might be doing something. It seems neuroscientists are good at discovering low level detail and perhaps not so good in general (visual cortex being somewhat of an exception) at putting the pieces together to suggest high level operations. ANNs seem complementary in that while their low level details are little like the brain, the connectionist architectures can be comparable, and we do know the top down operation (even though interpretation is an issue, more for some ANNs than others). If we assume that the cortex is doing some type of prediction error minimization then it's likely to have found similar solutions to an ANN in cases where problem and connectivity are similar.
  - foobarqux 3y ago
    Again this is just a hypothesis and nothing has come from it despite the fact that it is very old. There isn't any good reason for it to be true and no one has done anything to show that it is.
    - HarHarVeryFunny 3y ago
      Not sure what you're referring to - what's a hypothesis/old ?
      - foobarqux 3y ago
        Pretty much any claim that the brain (outside the visual cortex) work anything like the way neural nets do.

amelius 3y ago

But (how) does the human brain perform backpropagation?

HarHarVeryFunny 3y ago
It probably doesn't use backpropagation of gradients. Instead, the cortex appears to be a prediction engine that uses error feedback (perceptual reality vs prediction) to minimize prediction errors in a conceptually similar type of way. If every "layer" (cortical patch) is doing it's own prediction and receiving feedback, then you don't need any error propagation from one layer to the next.
- marcosdumay 3y ago
  You just can't go changing the layers of a neural network independently of each other if you are doing guided optimization. For a start, it's not even a given if increasing some weight will increase or decrease the result. The neurons are all tightly coupled. You can do it on unguided optimization, what is one of the reasons I strongly suspect our brains use something similar to simulated annealing.
  - HarHarVeryFunny 3y ago
    Sure, we don't know the exact details (Geoff Hinton spent much of his career trying to answer this question), but at the big picture level it does seem clear that the cortex is a prediction engine that minimizes prediction errors by feedback, and most likely does so in a localized way. Exactly how these prediction updates work is unknown. Could you expand a bit on how you think simulated annealing could work?
    - deleted 3y ago
      [deleted]
    - marcosdumay 3y ago
      Hum... I have no model for what actually happens. It's just that simulated annealing appears quite often on the biologic world, we never found any coordination communication channel that could create something more "intelligent" (AFAIK, there are just a few "neighborhood to global" analog signals available), and there isn't a multiplicity of attempts interacting with each other (again, there is no channel for that) that would lead to a parallel evolutionary strategy. Also, when we learn a mechanical task, our errors tend to cool down around our successes, instead of following a direction. But again, I have no idea of any mechanism that is there. It's more because all the others seem to have been eliminated.
canjobear 3y ago
Approximately, via predictive coding https://arxiv.org/abs/2006.04182 https://arxiv.org/abs/2006.04182

foobarqux 3y ago

This is just false. Outside of the visual cortex there isn't any evidence that brains work anything like GPT-4 or neural nets. Producing the same outputs isn't evidence of anything. At best you have just stated a hypothesis of how the brain might work, not any actual evidence supporting it.

HarHarVeryFunny 3y ago
Sure - you can't just look at how an ANN works and assume that's how the brain does that too, but the ANN's operation can act as inspiration to suggest or confirm the way the brain might be doing something. It seems neuroscientists are good at discovering low level detail and perhaps not so good in general (visual cortex being somewhat of an exception) at putting the pieces together to suggest high level operations. ANNs seem complementary in that while their low level details are little like the brain, the connectionist architectures can be comparable, and we do know the top down operation (even though interpretation is an issue, more for some ANNs than others). If we assume that the cortex is doing some type of prediction error minimization then it's likely to have found similar solutions to an ANN in cases where problem and connectivity are similar.
- foobarqux 3y ago
  Again this is just a hypothesis and nothing has come from it despite the fact that it is very old. There isn't any good reason for it to be true and no one has done anything to show that it is.
  - HarHarVeryFunny 3y ago
    Not sure what you're referring to - what's a hypothesis/old ?
    - foobarqux 3y ago
      Pretty much any claim that the brain (outside the visual cortex) work anything like the way neural nets do.