5 ms·

Computational Model Reveals How the Brain Manages Short-Term Memories

92 points by prostoalex 6y ago

babesh 6y ago
What do these inhibitory neurons do to manage short-term memories?
- jand 6y ago
  Optimistic guess: This information can be found in the paywall'ed article at [1] > Dissecting our recurrent neural network model revealed strong inhibitory-to-inhibitory connections underlying a disinhibitory microcircuit as a critical component for long neuronal timescales and WM maintenance If the article is worth the money, i would expect it to go into detail on it. [1] https://www.nature.com/articles/s41593-020-00753-w https://www.nature.com/articles/s41593-020-00753-w
  - greyface- 6y ago
    https://sci-hub.se/https://www.nature.com/articles/s41593-020-00753-w https://sci-hub.se/https://www.nature.com/articles/s41593-02...
- subroutine 6y ago
  This is explained in the Nature article (summary on page 133) and shown as a diagrams in figure 6... > In summary, these findings imply that robust inhibition of oppositely tuned inhibitory subpopulations is critical for memory maintenance in our RNN model. For example, a positive cue stimulus activates the inhibitory and excitatory subgroups selective for that stimulus and deactivates the negative stimulus subgroups. During the delay period, the inhibition strength between these two inhibitory subgroups dictates the stability of the cue-specific activity patterns generated during the stimulus window. The positive feedback provided by the similarly tuned excitatory neurons sustains the stimulus-specific activity of the inhibitory subgroups.
  - goldenshale 6y ago
    My intuitive understanding of these findings is that short term memory works by maintaining a consistent firing pattern in a particular region of the prefrontal cortex. By default a given firing pattern might activate other patterns because neurons tend to be a part of multiple potential pattern groups, and so left unchecked it could result in an unstable, random walk type of sequence. To stay in the attractor of one memory though, inhibitory neurons are used to turn off everything that isn't a part of the memory, thereby allowing it to remain in a more steady state.
    - subroutine 6y ago
      To further clarify, I believe the authors are advocating a WM model where an excitatory subnet (activated by a Positive cue) sustains its firing (in the absence of that cue; the Delay period) by activating inhibitors of its own inhibitors. (these inhibitory subnets, like goldenshale mentions, also seem to inhibit excitatory subnets, diffusely). model diagram: https://i.ibb.co/nrh1pVJ/figs.png https://i.ibb.co/nrh1pVJ/figs.png
earthboundkid 6y ago
> Kim and Sejnowski found that good working memory required both that long-timescale neurons be prevalent, and that connections between inhibitory neurons–which suppress brain activity–be strong. When they altered the strength of connections between these inhibitory neurons in their model, the researchers could change how well the model performed on the working memory test as well as the timescale of the pertinent neurons. Neuroscience is so laughably primitive. Imagine saying you were close to understanding computers because you know which capacitors you really can’t cut out. It’s just absurd how little actual knowledge of the mind we have, as opposed to anatomical facts or behavioral generalizations.
- lamename 6y ago
  As a neuroscientist, I can tell you with high confidence that many if not most neuroscientists would agree with you. You might also be interested in "Could a Neuroscientist Understand a Microprocessor?" https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005268 https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...
  - amirkdv 6y ago
    I can't recommend this paper enough to anyone interested in neuroscience. It's a thoughtfully executed set of experiments that IMHO should be part of every computational neuroscientist's training.
  - ohazi 6y ago
    This paper was thoroughly entertaining. Thanks!
- l33tman 6y ago
  I guess neuroscience mostly compares to the condensed matter / semiconductor field, in the computer analogy. They know everything about the transistors but guess at why they're connected like they are. And some in their respective fields will also be very interested in the high-level connections although maybe with less depth.. Unfortunately, in the neuroscience area, there is not yet any analogy to a computer programmer.
h-1 6y ago
Anyone can point out that autistic, schizophrenic or very old people lack certain inhibitions. But it's downright fascinating to think it may be explained by dulled inhibitory neurons or hypersensitive "regular" ones. That could be an easy fix!
- trevyn 6y ago
  >That could be an easy fix! Famous last words?
  - cortezdecoy 6y ago
    In one sense, almost certainly. But in another sense, any plausible fix is easy compared to the current situation.
    - trevyn 6y ago
      The sense I intended was that it may indeed be an easy fix, but second- third- etc- order effects may actually make us all worse off in ways we never considered.
bra-ket 6y ago
here's a free version https://scihubtw.tw/https://www.nature.com/articles/s41593-020-00753-w https://scihubtw.tw/https://www.nature.com/articles/s41593-0... Looks like another paper on how everything in the brain can be tied to RNN or GPT2 or some other trendy ML model. 'Cure cancer with deep learning' style. Before something like RNN can be used in a model of biological memory it has to respect the constraints of biological memory. Taking an arbitrary model and twisting the arm of data to fit it makes for some hard sale. Claiming the findings somehow elucidate memory impairments in schizophrenia and autism (which are not even close to being understood), makes it even more look like bad science.
- subroutine 6y ago
  I agree with this sentiment. To be fair though, the authors did attempt to generate a biologically plausible RNN, which they published in a previous article... https://www.pnas.org/content/pnas/116/45/22811.full.pdf https://www.pnas.org/content/pnas/116/45/22811.full.pdf I'm not fully sold on this model, but the article is fairly dense and I dont fully understand it. (Sejnowski, the senior author, was on my dissertation committee; he fell asleep during my defense lol)
  - bra-ket 6y ago
    Oh god, what he calls constraints are the "spiking nature of biological neurons", then he happily proceeds, just like you'd guess, to create this abomination of 'spiking RNN' and train this pretty much plain vanilla RNN via backpropagation, completely disregarding the energy spent, learning mechanism, and actual biological plausibility or relevance of this horrendous architecture as memory model. The first author knows just enough to be dangerous. >Sejnowski fell asleep it surely looks like it
    - subroutine 6y ago
      Dropping a link below in case you're interested - I have a side-project that takes a more literal approach to modeling biologically plausible neural networks (MCMC simulations inside 3D models)... https://github.com/bradmonk/plasticity https://github.com/bradmonk/plasticity
whymauri 6y ago
Glad to see more work in the 'stable memories' camp win out against the 'the brain is too stochastic and therefore unstable' theories. As a complement to this work on short-term memory, here's one of my favorite papers on long-term memories! https://science.sciencemag.org/content/365/6455/821.abstract https://science.sciencemag.org/content/365/6455/821.abstract
mtrycz2 6y ago
> Model reveals No, a model models, does not (and can not) reveal anything.
- glial 6y ago
  It can predict, but that’s a different thing entirely.
- hackinthebochs 6y ago
  A model based on some core principles or constraints of the natural system will often solve the problem by discovering the same structure used by nature. And so facts of the model can reveal facts of the natural system. Of course such discoveries need to be validated against the real thing.

92 points by prostoalex 6y ago

babesh 6y ago

What do these inhibitory neurons do to manage short-term memories?

jand 6y ago
Optimistic guess: This information can be found in the paywall'ed article at [1] > Dissecting our recurrent neural network model revealed strong inhibitory-to-inhibitory connections underlying a disinhibitory microcircuit as a critical component for long neuronal timescales and WM maintenance If the article is worth the money, i would expect it to go into detail on it. [1] https://www.nature.com/articles/s41593-020-00753-w https://www.nature.com/articles/s41593-020-00753-w
- greyface- 6y ago
  https://sci-hub.se/https://www.nature.com/articles/s41593-020-00753-w https://sci-hub.se/https://www.nature.com/articles/s41593-02...
subroutine 6y ago
This is explained in the Nature article (summary on page 133) and shown as a diagrams in figure 6... > In summary, these findings imply that robust inhibition of oppositely tuned inhibitory subpopulations is critical for memory maintenance in our RNN model. For example, a positive cue stimulus activates the inhibitory and excitatory subgroups selective for that stimulus and deactivates the negative stimulus subgroups. During the delay period, the inhibition strength between these two inhibitory subgroups dictates the stability of the cue-specific activity patterns generated during the stimulus window. The positive feedback provided by the similarly tuned excitatory neurons sustains the stimulus-specific activity of the inhibitory subgroups.
- goldenshale 6y ago
  My intuitive understanding of these findings is that short term memory works by maintaining a consistent firing pattern in a particular region of the prefrontal cortex. By default a given firing pattern might activate other patterns because neurons tend to be a part of multiple potential pattern groups, and so left unchecked it could result in an unstable, random walk type of sequence. To stay in the attractor of one memory though, inhibitory neurons are used to turn off everything that isn't a part of the memory, thereby allowing it to remain in a more steady state.
  - subroutine 6y ago
    To further clarify, I believe the authors are advocating a WM model where an excitatory subnet (activated by a Positive cue) sustains its firing (in the absence of that cue; the Delay period) by activating inhibitors of its own inhibitors. (these inhibitory subnets, like goldenshale mentions, also seem to inhibit excitatory subnets, diffusely). model diagram: https://i.ibb.co/nrh1pVJ/figs.png https://i.ibb.co/nrh1pVJ/figs.png

earthboundkid 6y ago

> Kim and Sejnowski found that good working memory required both that long-timescale neurons be prevalent, and that connections between inhibitory neurons–which suppress brain activity–be strong. When they altered the strength of connections between these inhibitory neurons in their model, the researchers could change how well the model performed on the working memory test as well as the timescale of the pertinent neurons. Neuroscience is so laughably primitive. Imagine saying you were close to understanding computers because you know which capacitors you really can’t cut out. It’s just absurd how little actual knowledge of the mind we have, as opposed to anatomical facts or behavioral generalizations.

lamename 6y ago
As a neuroscientist, I can tell you with high confidence that many if not most neuroscientists would agree with you. You might also be interested in "Could a Neuroscientist Understand a Microprocessor?" https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005268 https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...
- amirkdv 6y ago
  I can't recommend this paper enough to anyone interested in neuroscience. It's a thoughtfully executed set of experiments that IMHO should be part of every computational neuroscientist's training.
- ohazi 6y ago
  This paper was thoroughly entertaining. Thanks!
l33tman 6y ago
I guess neuroscience mostly compares to the condensed matter / semiconductor field, in the computer analogy. They know everything about the transistors but guess at why they're connected like they are. And some in their respective fields will also be very interested in the high-level connections although maybe with less depth.. Unfortunately, in the neuroscience area, there is not yet any analogy to a computer programmer.

h-1 6y ago

Anyone can point out that autistic, schizophrenic or very old people lack certain inhibitions. But it's downright fascinating to think it may be explained by dulled inhibitory neurons or hypersensitive "regular" ones. That could be an easy fix!

trevyn 6y ago
>That could be an easy fix! Famous last words?
- cortezdecoy 6y ago
  In one sense, almost certainly. But in another sense, any plausible fix is easy compared to the current situation.
  - trevyn 6y ago
    The sense I intended was that it may indeed be an easy fix, but second- third- etc- order effects may actually make us all worse off in ways we never considered.

bra-ket 6y ago

here's a free version https://scihubtw.tw/https://www.nature.com/articles/s41593-020-00753-w https://scihubtw.tw/https://www.nature.com/articles/s41593-0... Looks like another paper on how everything in the brain can be tied to RNN or GPT2 or some other trendy ML model. 'Cure cancer with deep learning' style. Before something like RNN can be used in a model of biological memory it has to respect the constraints of biological memory. Taking an arbitrary model and twisting the arm of data to fit it makes for some hard sale. Claiming the findings somehow elucidate memory impairments in schizophrenia and autism (which are not even close to being understood), makes it even more look like bad science.

subroutine 6y ago
I agree with this sentiment. To be fair though, the authors did attempt to generate a biologically plausible RNN, which they published in a previous article... https://www.pnas.org/content/pnas/116/45/22811.full.pdf https://www.pnas.org/content/pnas/116/45/22811.full.pdf I'm not fully sold on this model, but the article is fairly dense and I dont fully understand it. (Sejnowski, the senior author, was on my dissertation committee; he fell asleep during my defense lol)
- bra-ket 6y ago
  Oh god, what he calls constraints are the "spiking nature of biological neurons", then he happily proceeds, just like you'd guess, to create this abomination of 'spiking RNN' and train this pretty much plain vanilla RNN via backpropagation, completely disregarding the energy spent, learning mechanism, and actual biological plausibility or relevance of this horrendous architecture as memory model. The first author knows just enough to be dangerous. >Sejnowski fell asleep it surely looks like it
  - subroutine 6y ago
    Dropping a link below in case you're interested - I have a side-project that takes a more literal approach to modeling biologically plausible neural networks (MCMC simulations inside 3D models)... https://github.com/bradmonk/plasticity https://github.com/bradmonk/plasticity

whymauri 6y ago

Glad to see more work in the 'stable memories' camp win out against the 'the brain is too stochastic and therefore unstable' theories. As a complement to this work on short-term memory, here's one of my favorite papers on long-term memories! https://science.sciencemag.org/content/365/6455/821.abstract https://science.sciencemag.org/content/365/6455/821.abstract

mtrycz2 6y ago

> Model reveals No, a model models, does not (and can not) reveal anything.

glial 6y ago
It can predict, but that’s a different thing entirely.
hackinthebochs 6y ago
A model based on some core principles or constraints of the natural system will often solve the problem by discovering the same structure used by nature. And so facts of the model can reveal facts of the natural system. Of course such discoveries need to be validated against the real thing.