3 ms·
Chomsky's problem here has nothing to do with his politics, but unfortunately a lot to do with his long-held position in the Nature/Nurture debate - a position
by thomassmith65 1y ago
Chomsky's problem here has nothing to do with his politics, but unfortunately a lot to do with his long-held position in the Nature/Nurture debate - a position that is undermined by the ability of LLMs to learn language without hardcoded grammatical rules:
Chomsky introduced his theory of language acquisition, according to which children have an inborn quality of being biologically encoded with a universal grammar
https://psychologywriting.com/skinner-and-chomsky-on-nature-vs-nurture/ https://psychologywriting.com/skinner-and-chomsky-on-nature-...
- js8 1y agoI don't see how the two things are related. Whether acquisition of human language is nature or nurture - it is still learning of some sort. Yes, maybe we can reproduce that learning process in LLMs, but that doesn't mean the LLMs imitate only the nurture part (might as well be just finetuning), and not the nature part. An airplane is not an explanation for a bird's flight.
- thomassmith65 1y agoThe great breakthrough in AI turned out to be LLMs. Nature, for an LLM, is its design: graph, starting weights, etc. Environment, for an LLM, is what happens during training. LLMs are capable of learning grammar entirely from their environment, which suggests that infants are too, which is bad for Chomsky's position that the basics of grammar are baked into human DNA.
- suddenlybananas 1y agoLLMs require vastly more data than humans and still struggle with some more esoteric grammatical rules like parasitic gaps. The fact grammar can be approximated given trillions of words doesn't explain how babies learn language from a much more modest dataset.
- thomassmith65 1y agoIt's not that the invention of LLMs conclusively disproves Chomksy's position. However, we now have a proof-of-concept that a computer can learn grammar in a sophisticated way, from the ground up. We have yet to code something procedural that approaches the same calibre via a hard-coded universal grammar. That may not obliterate Chomksy's position, but it looks bad.
- suddenlybananas 1y agoThat's not the goal of generative linguistics though; it's not an engineering project.
- thomassmith65 1y agoThe problem encompasses not just biology and information technology, but also linguistics. Even if LLMs say nothing about biology, they do tell us something about the nature of language itself. Again, that LLMs can learn to compose sophisticated texts from training alone does not close the case on Chomsky's position. However, it is a piece of evidence against it. It does suggest, by Occam's razor, that a hardwired universal grammar is the lesser theory.
- suddenlybananas 1y agoHow do LLMs explain how 5 year olds respect island constraints?
- thomassmith65 1y agoI don't have the domain knowledge to discuss that.
- suddenlybananas 1y agoIf you don't know what a syntactic island is, perhaps you're not the best judge of the plausibility of a linguistic theory.