4 ms·

The generation of natural language is an aspect of human cognition, and I'm not aware of any better model for that than current statistical LLMs. The papers map

The generation of natural language is an aspect of human cognition, and I'm not aware of any better model for that than current statistical LLMs. The papers mapping between EEG/fMRI/etc. and LLM activations have been generally oversold so far, but it's active area of research for good reason. I'm not saying LLMs are a particularly good model, just that everything else is currently worse. This includes Chomsky's formal grammars, which fail to capture the ways humans actually use language per Norvig's many examples. Do you disagree? If so, what model is better and why?

foldr 9mo ago
I’m not really sure what you’re getting at. Could you point to some papers exemplifying the kind of work that you’re thinking of? Of course there are lots of people training LLMs and other statistical models on EEG data, but that does not show that, say, GPT-5, is a good model of any aspect of human cognition. Chomsky, of course, never attempted to model the generation of natural language and was interested in a different set of problems, so LLMs are not really a competitor in that sense anyway (even if you take the dubious step of accepting them as scientific models). I certainly don’t agree with Norvig, but he doesn’t really understand the basics of what Chomsky is trying to do, so there is not much to respond to. To give three specific examples, he (i) is confused in thinking that Gold’s theorem has anything to do with Chomsky’s arguments, (ii) appears to think that Chomsky studied the “generation of language” (because he he’s read so little of Chomsky’s work that he doesn’t know what a “generative grammar” is), and (iii) believes that Chomsky thinks that natural languages are formal languages in which every possible sentence is either in the language or not (again because he’s barely read anything that Chomsky wrote since the 1950s). Then, just to make absolutely sure not to be taken seriously, he compares Chomsky to Bill O’Reilly! On point (iii), see http://www.linguistics.berkeley.edu/~syntax-circle/syntax-group/spr06/chomsky1990.pdf http://www.linguistics.berkeley.edu/~syntax-circle/syntax-gr..., and the last complete paragraph of p. 145.
- D-Machine 9mo ago
  This comment and GP comment are why the word "causal model" is needed. LLMs are predictive* models of human language, but they are not causal models of language. If you believe that some of human cognition is linguistic (even if e.g. inner monologue and spoken language are just the surface of deeper more unconscious processes), then, yes, we might say LLMs can predictively model some aspects of human cognition, but, again, they are certainly not causal models, and they are not predictive models of human cognition generally (as cognition is clearly far, far more than linguistic). * I avoid calling LLMs "statistical" because they really aren't even that. They are not calibrated, and including a softmax and log-loss in things doesn't magically make your model statistical (especially since ad-hoc regularization methods, other loss functions and simplex mappings, e.g. sparsemax, often work better and then violate the assumptions that are needed to prove these things are behaving statistically). LLMs really are more accurately just doing (very, very fancy and impressive) curve/manifold-fitting.
  - foldr 9mo ago
    They are not predictive models in the domains Chomsky investigated. LLMs make no predictions about, say, when non-surface quantifier scope should or should not be possible, or what should or shouldn’t be a wh-island. They are predictive in a sense that’s largely irrelevant to cognitive science. (Trying to guess what words might come after some other words isn’t a problem in cognitive science.)
    - D-Machine 9mo ago
      Correct, LLMs are predictive also only in a narrow sense!
    - tripletao 9mo ago
      "What should or shouldn’t be a wh-island" is literally a statement of "what words might come after some other words"! An LLM encodes billions of such statements, just unfortunately in a quantity and form that makes them incomprehensible to an unaided human. That part is strictly worse; but the LLM's statements model language well enough to generate it, and that part is strictly better. As I read Norvig's essay, it's about that tradeoff, of whether a simple and comprehensible but inaccurate model shows more promise than a model that's incomprehensible except in statistical terms with the aid of a computer, but far more accurate. I understand there's a large group of people who think Norvig is wrong or incoherent; but when those people have no accomplishments except within the framework they themselves have constructed, what am I supposed to think? Beyond that, if I have a model that tells me whether a sentence is valid, then I can always try different words until I find one that makes it valid. Any sufficiently good model is thus capable of generation. Chomsky never proposed anything capable of that; but that just means his models were bad, not that he was working on a different task. As to the relationship between signals from biological neurons and ANN activations, I mean something like the paper linked below, whose authors write: > Thus, even though the goal of contemporary AI is to improve model performance and not necessarily to build models of brain processing, this endeavor appears to be rapidly converging on architectures that might capture key aspects of language processing in the human mind and brain. https://www.biorxiv.org/content/10.1101/2020.06.26.174482v3.full.pdf https://www.biorxiv.org/content/10.1101/2020.06.26.174482v3.... I emphasize again that I believe these results have been oversold in the popular press, but the idea that an ANN trained on brain output (including written language) might provide insight into the physical, causal structure of the brain is pretty mainstream now.
D-Machine 9mo ago
Also, in case you missed the recent big thread, fMRI has taught us almost nothing due to its serious limitations and various measurement and design issues in the field. IMO it is way too slow and clunky to ever yield insights into something as fast as linguistic thought. https://news.ycombinator.com/item?id=46288415 https://news.ycombinator.com/item?id=46288415

foldr 9mo ago

I’m not really sure what you’re getting at. Could you point to some papers exemplifying the kind of work that you’re thinking of? Of course there are lots of people training LLMs and other statistical models on EEG data, but that does not show that, say, GPT-5, is a good model of any aspect of human cognition. Chomsky, of course, never attempted to model the generation of natural language and was interested in a different set of problems, so LLMs are not really a competitor in that sense anyway (even if you take the dubious step of accepting them as scientific models). I certainly don’t agree with Norvig, but he doesn’t really understand the basics of what Chomsky is trying to do, so there is not much to respond to. To give three specific examples, he (i) is confused in thinking that Gold’s theorem has anything to do with Chomsky’s arguments, (ii) appears to think that Chomsky studied the “generation of language” (because he he’s read so little of Chomsky’s work that he doesn’t know what a “generative grammar” is), and (iii) believes that Chomsky thinks that natural languages are formal languages in which every possible sentence is either in the language or not (again because he’s barely read anything that Chomsky wrote since the 1950s). Then, just to make absolutely sure not to be taken seriously, he compares Chomsky to Bill O’Reilly! On point (iii), see http://www.linguistics.berkeley.edu/~syntax-circle/syntax-group/spr06/chomsky1990.pdf http://www.linguistics.berkeley.edu/~syntax-circle/syntax-gr..., and the last complete paragraph of p. 145.

D-Machine 9mo ago
This comment and GP comment are why the word "causal model" is needed. LLMs are predictive* models of human language, but they are not causal models of language. If you believe that some of human cognition is linguistic (even if e.g. inner monologue and spoken language are just the surface of deeper more unconscious processes), then, yes, we might say LLMs can predictively model some aspects of human cognition, but, again, they are certainly not causal models, and they are not predictive models of human cognition generally (as cognition is clearly far, far more than linguistic). * I avoid calling LLMs "statistical" because they really aren't even that. They are not calibrated, and including a softmax and log-loss in things doesn't magically make your model statistical (especially since ad-hoc regularization methods, other loss functions and simplex mappings, e.g. sparsemax, often work better and then violate the assumptions that are needed to prove these things are behaving statistically). LLMs really are more accurately just doing (very, very fancy and impressive) curve/manifold-fitting.
- foldr 9mo ago
  They are not predictive models in the domains Chomsky investigated. LLMs make no predictions about, say, when non-surface quantifier scope should or should not be possible, or what should or shouldn’t be a wh-island. They are predictive in a sense that’s largely irrelevant to cognitive science. (Trying to guess what words might come after some other words isn’t a problem in cognitive science.)
  - D-Machine 9mo ago
    Correct, LLMs are predictive also only in a narrow sense!
  - tripletao 9mo ago
    "What should or shouldn’t be a wh-island" is literally a statement of "what words might come after some other words"! An LLM encodes billions of such statements, just unfortunately in a quantity and form that makes them incomprehensible to an unaided human. That part is strictly worse; but the LLM's statements model language well enough to generate it, and that part is strictly better. As I read Norvig's essay, it's about that tradeoff, of whether a simple and comprehensible but inaccurate model shows more promise than a model that's incomprehensible except in statistical terms with the aid of a computer, but far more accurate. I understand there's a large group of people who think Norvig is wrong or incoherent; but when those people have no accomplishments except within the framework they themselves have constructed, what am I supposed to think? Beyond that, if I have a model that tells me whether a sentence is valid, then I can always try different words until I find one that makes it valid. Any sufficiently good model is thus capable of generation. Chomsky never proposed anything capable of that; but that just means his models were bad, not that he was working on a different task. As to the relationship between signals from biological neurons and ANN activations, I mean something like the paper linked below, whose authors write: > Thus, even though the goal of contemporary AI is to improve model performance and not necessarily to build models of brain processing, this endeavor appears to be rapidly converging on architectures that might capture key aspects of language processing in the human mind and brain. https://www.biorxiv.org/content/10.1101/2020.06.26.174482v3.full.pdf https://www.biorxiv.org/content/10.1101/2020.06.26.174482v3.... I emphasize again that I believe these results have been oversold in the popular press, but the idea that an ANN trained on brain output (including written language) might provide insight into the physical, causal structure of the brain is pretty mainstream now.

D-Machine 9mo ago

Also, in case you missed the recent big thread, fMRI has taught us almost nothing due to its serious limitations and various measurement and design issues in the field. IMO it is way too slow and clunky to ever yield insights into something as fast as linguistic thought. https://news.ycombinator.com/item?id=46288415 https://news.ycombinator.com/item?id=46288415