5 ms·
From the conclusion: > First, the fact that language models can be trained on large amounts of text data and can generate human-like language without any expli
by zvmaz 3y ago
From the conclusion:
> First, the fact that language models can be trained on large amounts of text data
and can generate human-like language without any explicit instruction on gram-
mar or syntax suggests that language may not be as biologically determined as
Chomsky has claimed. Instead, it suggests that language may be learned and
developed through exposure to language and interactions with others.
I'm not a linguist nor a cognitive scientist, but this seems so problematic that I am not sure that I read it correctly. For example, how is the fact that language models "work" contradict the innateness of language in humans?
- taeric 3y agoIt contradicts the idea that you have to teach language using syntax and grammar. Which... I confess I thought was already not believed? We certainly aren't teaching kids in the home how to decline and conjugate words. Are we? (Similarly, languages that have gender are typically just picked up by usage, not necessarily ingrained by reasoning. Which leads to the obvious bad results when people think that there was solid reasoning on those choices, in the first place.)
- pas 3y agoyep. Chomsky used the term "universal grammar", but it's much more likely a universal abstract semantics++ thing (coupled with sound processing stuff, plus repetition, plus a bunch of other stuff in the brain that helps learning in general). how much is innate, what exactly does that mean, all good questions. and of course raw intelligence (pattern matching, strategizing, learning, adaptiveness, modeling, ability to form a sort of consistent and goal-orientedly useful predictive model of the world based on inputs, and goal-oriented control of behavior based on these aforementioned models) by definition can learn language. and of course it's a strange question of are LLMs intelligent in this sense despite lacking goals?
- bbor 3y agoYeah perhaps that’s causing confusion for science enthusiasts new to this debate - Chomsky is definitely not talking about what a lay person would call “Grammar”. Your point is one of his main (implicit) supports though: on the face of it, it seems absolutely insane that a child could pick up complex linguistic concepts in a few short years, with many orders of magnitude less data than a LLM needs to reach the same capabilities
- taeric 3y agoIt is rather amusing how poorly industries get their jargon out into the wild. Such that it isn't that surprising to me, that this could be the crux of some misunderstanding here.
- meepmorp 3y ago> For example, how is the fact that language models "work" contradict the innateness of language in humans? It doesn't. Also, the author doesn't seem to actually understand Chomsky's writing about language, because learning language via exposure is how humans learn languages and he fucking mentions that in his writing on the subject. UG (universal grammar) is the purported facility in the human brain which makes language possible - it has a innate structure, but it learns particular languages from exposure. Chomsky doesn't state exactly what that structure is because he doesn't know - figuring that out is goal of his work.