6 ms·

Brainwaves Encode the Grammar of Human Language

104 points by dnetesn 8y ago

caublestone 8y ago
I wonder if the structure of brain waves is a result of a gravity time crystal. https://arxiv.org/pdf/1708.05014.pdf https://arxiv.org/pdf/1708.05014.pdf
- deleted 8y ago
  [deleted]
- kaghyu1667 8y ago
  Do you even know what that means?
  - blattimwind 8y ago
    I think microgravitational brain oscillations related to intermittent contacting of crystal-based planars should be explored more deeply.
    - anon2775 8y ago
      Sorry, but I'm going to need a lot more weed before any of this makes any sense.
laretluval 8y ago
More like, brainwaves can encode grammatical structure in one particular model.
carapace 8y ago
This article reminded me of the (poorly named) "Feynman Machine": https://arxiv.org/abs/1609.03971 https://arxiv.org/abs/1609.03971 > Efforts at understanding the computational processes in the brain have met with limited success, despite their importance and potential uses in building intelligent machines. We propose a simple new model which draws on recent findings in Neuroscience and the Applied Mathematics of interacting Dynamical Systems. The Feynman Machine is a Universal Computer for Dynamical Systems, analogous to the Turing Machine for symbolic computing, but with several important differences. We demonstrate that networks and hierarchies of simple interacting Dynamical Systems, each adaptively learning to forecast its evolution, are capable of automatically building sensorimotor models of the external and internal world. We identify such networks in mammalian neocortex, and show how existing theories of cortical computation combine with our model to explain the power and flexibility of mammalian intelligence. These findings lead directly to new architectures for machine intelligence. A suite of software implementations has been built based on these principles, and applied to a number of spatiotemporal learning tasks. YMMV
- dmos62 8y ago
  Sounds interesting, but this 2016 paper has only one citation. I'm reading it now and this paragraph from the introduction compares it to traditional Deep Learning networks: > Our artificial Feynman Machines have several interesting properties which distinguish them from existing Deep Learning and similar systems. In particular, due to the much higher density and locality of processing, a Feynman Machine-based system can perform at least comparably while dramatically reducing the footprint in computational power, training data and fine-tuning. Feynman Machines can be arbitrarily split into modules distributed across clusters and the Internet, and systems running on low power devices such as phones can be dynamically and robustly augmented using low-bandwidth connections to larger networks running in the cloud. Models can be trained on powerful infrastructure and transferred for operation and further custom learning on smaller devices. Importantly, the same architecture can be used as a component in unsupervised, semi-supervised, fully supervised and reinforcement learning contexts. A variant - the Routed Predictive Hierarchy - is described, which allows a Feynman Machine to directly control a traditional Deep Learning network by switching it to use spatiotemporally-selected subnetworks. This is the abstract of the previous paper that detailed the theoretical basis [0]: > Reverse engineering the brain is proving difficult, perhaps impossible. While many believe that this is just a matter of time and effort, a different approach might help. Here, we describe a very simple idea which explains the power of the brain as well as its structure, exploiting complex dynamics rather than abstracting it away. Just as a Turing Machine is a Universal Digital Computer operating in a world of symbols, we propose that the brain is a Universal Dynamical Systems Modeller, evolved bottom-up (itself using nested networks of interconnected, self-organised dynamical systems) to prosper in a world of dynamical systems. Recent progress in Applied Mathematics has produced startling evidence of what happens when abstract Dynamical Systems interact. Key latent information describing system A can be extracted by system B from very simple signals, and signals can be used by one system to control and manipulate others. Using these facts, we show how a region of the neocortex uses its dynamics to intrinsically "compute" about the external and internal world. Building on an existing "static" model of cortical computation (Hawkins' Hierarchical Temporal Memory - HTM), we describe how a region of neocortex can be viewed as a network of components which together form a Dynamical Systems modelling module, connected via sensory and motor pathways to the external world, and forming part of a larger dynamical network in the brain. Empirical modelling and simulations of Dynamical HTM are possible with simple extensions and combinations of currently existing open source software. We list a number of relevant projects. Edit: Apparently, the project is called Ogma and they have a youtube channel. Here's their video [1] of a toy car learning to self-drive after a few laps of a person driving it with an xbox controller. It runs on a NanoPi Duo. Not sure if that's noteworthy or not. [0] https://arxiv.org/abs/1512.05245 https://arxiv.org/abs/1512.05245 [1] https://www.youtube.com/watch?v=gCuadXj9KDc https://www.youtube.com/watch?v=gCuadXj9KDc
sethbannon 8y ago
"Being able to learn and use grammar is unique to humans." While we haven't observed grammar in non-human animals yet, whether they can learn and use it is very much still an open question. "considering the rather limited number of experiments and the difficulty to design experiments that unequivocally demonstrate more complex rule learning, the question of what animals are able to do remains open." [1] "when we consider how animal vocalizations are analyzed, the types of stimuli and tasks that are used in artificial grammar learning experiments, the limited number of species examined, and the groups to which these belong, I argue that the currently available evidence is insufficient to arrive at firm conclusions concerning the limitations of animal grammatical abilities." [2] 1: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3367684/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3367684/ 2: https://link.springer.com/article/10.3758/s13423-016-1091-9 https://link.springer.com/article/10.3758/s13423-016-1091-9
- combatentropy 8y ago
  It's easy to be a skeptic forever. "I think we need more studies." Steven Pinker in his book The Language Instinct states that language is unique to humans. Whatever language animals have, it's like a pebble and the Chrysler Building. My cat can well tell me when he's hungry. But think of the richness of human language, which can express things in front of you and things a million miles away, things now and things past, their color, shape, activity. I can express whether something is, or hypothetically what it would be if, and I can express how I feel about the possibility.
  - flavor8 8y ago
    https://aeon.co/essays/if-elephants-arent-persons-yet-could-they-be-one-day https://aeon.co/essays/if-elephants-arent-persons-yet-could-...
  - starbeast 8y ago
    We really do need more studies to make any kind of statement about the uniqueness of grammar, given that we cannot communicate in a representative sample of animal languages in order to make this claim strongly and also that grammar can be quite simple, so we cannot infer this claim from the relative difficulty of the task.
dhairya 8y ago
"A key finding of the new study is that these artificial neural networks, when fed example sentences, give off patterns of energy that mimic what the brain does when it processes a sentence." I wish the article fleshed out more details about the research. It looks like they found a correlation between the output of a specific ANN that seems similar to observed neural oscillations in the brain. The causality claim in the article seems premature. Also curious if the neural encoding of grammar is consistent across different languages. So many questions about the specifics, but cool research for sure.
- gumboshoes 8y ago
  The research the article is based on is linked at the bottom of the article: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.2000663 https://journals.plos.org/plosbiology/article?id=10.1371/jou...
  - dhairya 8y ago
    Thanks for sharing that link! Totally missed it.
- deleted 8y ago
  [deleted]
Animats 8y ago
Is this a real result, or just speculation?
vonsydov 8y ago
Wow, this is completely unreal.
dhairya 8y ago
The experiment is fascinating. It looks like they developed a model (DORA) for learning relational reasoning. It turns out that model showed similar oscillatory activation patterns to observed cortical signals. As a comparison, they also looked at a fully connected RNN with a which did not produce oscillations. So the key research finding is that time-binding is a requisite to produce oscillations in neural networks. The paper's conclusions: "In sum, we remain relatively agnostic about the specific details of the required representational hierarchy because we do not yet know how to link the predicate calculus representations we use to natural language mental and cortical representations. What we are not agnostic about is the need for asynchronous time-based binding in order to produce oscillations in a neural network, as well as the need for representational hierarchy to produce the particular pattern of oscillations observed here and in" The paper on evaluating if NLP parsers with representational hierarchies reflect what the cortical functionality: "Our results naturally beg the converse question, as to whether any system with representational hierarchy could produce the oscillatory pattern of activation that [6] and DORA show. For example, could natural language processing (NLP) parsers, which feature representational hierarchy and were developed to specifically parse natural language in a machine, produce oscillations and the pattern seen in [6] and in our simulations? In principle, any system of hierarchy that is (de)compositional has the representational ingredients to encode units that could be fired in a sequence. However, we would argue that any given representational hierarchy could only produce oscillations if it were combined with time-based binding, which, as far as we know, no NLP system features. In DORA, time-based binding is the oscillation of activity throughout the network, which is part of the reason why the RNN did not show oscillatory activity nor the specific pattern from [6]. The representational structure of DORA (a (de)compositional role-filler binding predicate logic) is what makes the oscillations take the form that the data from [6] have. In terms of the specifics of the observed 1-2-4 Hz pattern, both [6] and our simulations are highly shaped by the word presentation rate of 250 ms/4 Hz. But without time-based binding, there is no mechanism to produce oscillations in a network, even in a NLP parser or other system with representational hierarchy."
phaedrus 8y ago
This reminds me of an idea I had when I was a freshman in college, in 2004 - 2005. I proposed a research project based on generalizing Markov chat bots to amplitude waves through neuron-like nodes. Basically, instead of discretely recognizing A -> B -> C but not A -> C -> B, the idea is the Markov node (or neuron) structure learned would respond with a sine-wave shaped pulse that initially rises and then goes negative, so that a chain of nodes "A -> B -> C" would still respond to a slightly out-of-order or different sequence, just not as strongly. Initially my undergraduate CS advisor was excited by my idea, but only, it turned out, in the context of using it in genetic sequencing (because buzzwords). I gave him my honest opinion that adding my contrivance could only perform worse that existing genetic sequencing algorithms; my intent was to propose and investigate a way the brain might reasonably be recognizing language. He was entirely uninterested in this, and I got too busy with college / work and never developed the idea beyond a toy demo. Maybe I should look at it again...
- patzal 8y ago
  It is depressing how many good thesis topics not getting accepted because of some kind of personal preference/agenda of the advisors. Similar story happened to me.
  - AlexCoventry 8y ago
    I think in this case it was probably more about funding opportunities.
- chewxy 8y ago
  So a variant of spiking neural networks?
buboard 8y ago
This is a model, a hypothesis, the title of nautilus makes it as it is a proven fact. "A mechanism for the cortical computation of hierarchical linguistic structure" is the original title. Reminds of another proposed model from Gyuri Buszaki about how natural brain rhythms might be used to encode sentences. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3553572/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3553572/
bra-ket 8y ago
Also see Pulvermuller, The Neuroscience of Language: https://www.amazon.com/Neuroscience-Language-Brain-Circuits-Serial/dp/0521793742 https://www.amazon.com/Neuroscience-Language-Brain-Circuits-...
tlow 8y ago
Very skeptical of the veracity of any claim that, "what the brain does when it processes a sentence." is at all universal across a wide sample of humans. What about, for example, differences in languages, culture, personal background, etc.
- ahartmetz 8y ago
  AFAIK there is scientific consensus that much of human language is hardwired. It's how infants can learn it so quickly.

104 points by dnetesn 8y ago

I wonder if the structure of brain waves is a result of a gravity time crystal. https://arxiv.org/pdf/1708.05014.pdf https://arxiv.org/pdf/1708.05014.pdf

deleted 8y ago
[deleted]
kaghyu1667 8y ago
Do you even know what that means?
- blattimwind 8y ago
  I think microgravitational brain oscillations related to intermittent contacting of crystal-based planars should be explored more deeply.
  - anon2775 8y ago
    Sorry, but I'm going to need a lot more weed before any of this makes any sense.

laretluval 8y ago

More like, brainwaves can encode grammatical structure in one particular model.

carapace 8y ago

This article reminded me of the (poorly named) "Feynman Machine": https://arxiv.org/abs/1609.03971 https://arxiv.org/abs/1609.03971 > Efforts at understanding the computational processes in the brain have met with limited success, despite their importance and potential uses in building intelligent machines. We propose a simple new model which draws on recent findings in Neuroscience and the Applied Mathematics of interacting Dynamical Systems. The Feynman Machine is a Universal Computer for Dynamical Systems, analogous to the Turing Machine for symbolic computing, but with several important differences. We demonstrate that networks and hierarchies of simple interacting Dynamical Systems, each adaptively learning to forecast its evolution, are capable of automatically building sensorimotor models of the external and internal world. We identify such networks in mammalian neocortex, and show how existing theories of cortical computation combine with our model to explain the power and flexibility of mammalian intelligence. These findings lead directly to new architectures for machine intelligence. A suite of software implementations has been built based on these principles, and applied to a number of spatiotemporal learning tasks. YMMV

dmos62 8y ago
Sounds interesting, but this 2016 paper has only one citation. I'm reading it now and this paragraph from the introduction compares it to traditional Deep Learning networks: > Our artificial Feynman Machines have several interesting properties which distinguish them from existing Deep Learning and similar systems. In particular, due to the much higher density and locality of processing, a Feynman Machine-based system can perform at least comparably while dramatically reducing the footprint in computational power, training data and fine-tuning. Feynman Machines can be arbitrarily split into modules distributed across clusters and the Internet, and systems running on low power devices such as phones can be dynamically and robustly augmented using low-bandwidth connections to larger networks running in the cloud. Models can be trained on powerful infrastructure and transferred for operation and further custom learning on smaller devices. Importantly, the same architecture can be used as a component in unsupervised, semi-supervised, fully supervised and reinforcement learning contexts. A variant - the Routed Predictive Hierarchy - is described, which allows a Feynman Machine to directly control a traditional Deep Learning network by switching it to use spatiotemporally-selected subnetworks. This is the abstract of the previous paper that detailed the theoretical basis [0]: > Reverse engineering the brain is proving difficult, perhaps impossible. While many believe that this is just a matter of time and effort, a different approach might help. Here, we describe a very simple idea which explains the power of the brain as well as its structure, exploiting complex dynamics rather than abstracting it away. Just as a Turing Machine is a Universal Digital Computer operating in a world of symbols, we propose that the brain is a Universal Dynamical Systems Modeller, evolved bottom-up (itself using nested networks of interconnected, self-organised dynamical systems) to prosper in a world of dynamical systems. Recent progress in Applied Mathematics has produced startling evidence of what happens when abstract Dynamical Systems interact. Key latent information describing system A can be extracted by system B from very simple signals, and signals can be used by one system to control and manipulate others. Using these facts, we show how a region of the neocortex uses its dynamics to intrinsically "compute" about the external and internal world. Building on an existing "static" model of cortical computation (Hawkins' Hierarchical Temporal Memory - HTM), we describe how a region of neocortex can be viewed as a network of components which together form a Dynamical Systems modelling module, connected via sensory and motor pathways to the external world, and forming part of a larger dynamical network in the brain. Empirical modelling and simulations of Dynamical HTM are possible with simple extensions and combinations of currently existing open source software. We list a number of relevant projects. Edit: Apparently, the project is called Ogma and they have a youtube channel. Here's their video [1] of a toy car learning to self-drive after a few laps of a person driving it with an xbox controller. It runs on a NanoPi Duo. Not sure if that's noteworthy or not. [0] https://arxiv.org/abs/1512.05245 https://arxiv.org/abs/1512.05245 [1] https://www.youtube.com/watch?v=gCuadXj9KDc https://www.youtube.com/watch?v=gCuadXj9KDc

sethbannon 8y ago

"Being able to learn and use grammar is unique to humans." While we haven't observed grammar in non-human animals yet, whether they can learn and use it is very much still an open question. "considering the rather limited number of experiments and the difficulty to design experiments that unequivocally demonstrate more complex rule learning, the question of what animals are able to do remains open." [1] "when we consider how animal vocalizations are analyzed, the types of stimuli and tasks that are used in artificial grammar learning experiments, the limited number of species examined, and the groups to which these belong, I argue that the currently available evidence is insufficient to arrive at firm conclusions concerning the limitations of animal grammatical abilities." [2] 1: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3367684/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3367684/ 2: https://link.springer.com/article/10.3758/s13423-016-1091-9 https://link.springer.com/article/10.3758/s13423-016-1091-9

combatentropy 8y ago
It's easy to be a skeptic forever. "I think we need more studies." Steven Pinker in his book The Language Instinct states that language is unique to humans. Whatever language animals have, it's like a pebble and the Chrysler Building. My cat can well tell me when he's hungry. But think of the richness of human language, which can express things in front of you and things a million miles away, things now and things past, their color, shape, activity. I can express whether something is, or hypothetically what it would be if, and I can express how I feel about the possibility.
- flavor8 8y ago
  https://aeon.co/essays/if-elephants-arent-persons-yet-could-they-be-one-day https://aeon.co/essays/if-elephants-arent-persons-yet-could-...
- starbeast 8y ago
  We really do need more studies to make any kind of statement about the uniqueness of grammar, given that we cannot communicate in a representative sample of animal languages in order to make this claim strongly and also that grammar can be quite simple, so we cannot infer this claim from the relative difficulty of the task.

dhairya 8y ago

"A key finding of the new study is that these artificial neural networks, when fed example sentences, give off patterns of energy that mimic what the brain does when it processes a sentence." I wish the article fleshed out more details about the research. It looks like they found a correlation between the output of a specific ANN that seems similar to observed neural oscillations in the brain. The causality claim in the article seems premature. Also curious if the neural encoding of grammar is consistent across different languages. So many questions about the specifics, but cool research for sure.

gumboshoes 8y ago
The research the article is based on is linked at the bottom of the article: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.2000663 https://journals.plos.org/plosbiology/article?id=10.1371/jou...
- dhairya 8y ago
  Thanks for sharing that link! Totally missed it.
deleted 8y ago
[deleted]

Animats 8y ago

Is this a real result, or just speculation?

vonsydov 8y ago

Wow, this is completely unreal.

dhairya 8y ago

The experiment is fascinating. It looks like they developed a model (DORA) for learning relational reasoning. It turns out that model showed similar oscillatory activation patterns to observed cortical signals. As a comparison, they also looked at a fully connected RNN with a which did not produce oscillations. So the key research finding is that time-binding is a requisite to produce oscillations in neural networks. The paper's conclusions: "In sum, we remain relatively agnostic about the specific details of the required representational hierarchy because we do not yet know how to link the predicate calculus representations we use to natural language mental and cortical representations. What we are not agnostic about is the need for asynchronous time-based binding in order to produce oscillations in a neural network, as well as the need for representational hierarchy to produce the particular pattern of oscillations observed here and in" The paper on evaluating if NLP parsers with representational hierarchies reflect what the cortical functionality: "Our results naturally beg the converse question, as to whether any system with representational hierarchy could produce the oscillatory pattern of activation that [6] and DORA show. For example, could natural language processing (NLP) parsers, which feature representational hierarchy and were developed to specifically parse natural language in a machine, produce oscillations and the pattern seen in [6] and in our simulations? In principle, any system of hierarchy that is (de)compositional has the representational ingredients to encode units that could be fired in a sequence. However, we would argue that any given representational hierarchy could only produce oscillations if it were combined with time-based binding, which, as far as we know, no NLP system features. In DORA, time-based binding is the oscillation of activity throughout the network, which is part of the reason why the RNN did not show oscillatory activity nor the specific pattern from [6]. The representational structure of DORA (a (de)compositional role-filler binding predicate logic) is what makes the oscillations take the form that the data from [6] have. In terms of the specifics of the observed 1-2-4 Hz pattern, both [6] and our simulations are highly shaped by the word presentation rate of 250 ms/4 Hz. But without time-based binding, there is no mechanism to produce oscillations in a network, even in a NLP parser or other system with representational hierarchy."

phaedrus 8y ago

This reminds me of an idea I had when I was a freshman in college, in 2004 - 2005. I proposed a research project based on generalizing Markov chat bots to amplitude waves through neuron-like nodes. Basically, instead of discretely recognizing A -> B -> C but not A -> C -> B, the idea is the Markov node (or neuron) structure learned would respond with a sine-wave shaped pulse that initially rises and then goes negative, so that a chain of nodes "A -> B -> C" would still respond to a slightly out-of-order or different sequence, just not as strongly. Initially my undergraduate CS advisor was excited by my idea, but only, it turned out, in the context of using it in genetic sequencing (because buzzwords). I gave him my honest opinion that adding my contrivance could only perform worse that existing genetic sequencing algorithms; my intent was to propose and investigate a way the brain might reasonably be recognizing language. He was entirely uninterested in this, and I got too busy with college / work and never developed the idea beyond a toy demo. Maybe I should look at it again...

patzal 8y ago
It is depressing how many good thesis topics not getting accepted because of some kind of personal preference/agenda of the advisors. Similar story happened to me.
- AlexCoventry 8y ago
  I think in this case it was probably more about funding opportunities.
chewxy 8y ago
So a variant of spiking neural networks?

buboard 8y ago

This is a model, a hypothesis, the title of nautilus makes it as it is a proven fact. "A mechanism for the cortical computation of hierarchical linguistic structure" is the original title. Reminds of another proposed model from Gyuri Buszaki about how natural brain rhythms might be used to encode sentences. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3553572/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3553572/

bra-ket 8y ago

Also see Pulvermuller, The Neuroscience of Language: https://www.amazon.com/Neuroscience-Language-Brain-Circuits-Serial/dp/0521793742 https://www.amazon.com/Neuroscience-Language-Brain-Circuits-...

tlow 8y ago

Very skeptical of the veracity of any claim that, "what the brain does when it processes a sentence." is at all universal across a wide sample of humans. What about, for example, differences in languages, culture, personal background, etc.

ahartmetz 8y ago
AFAIK there is scientific consensus that much of human language is hardwired. It's how infants can learn it so quickly.