3 ms·

I think it's pretty obvious that there is more than enough information in the training set of a typical human to learn grammar. Because we do. And people in dif

• dllthomas 3y ago
  That's the point, though. As you say there is obviously enough information to learn grammar given our starting point, but Chomsky's point is that there's not enough information to distinguish between all possible languages, so (paraphrasing from a decades-old recollection and probably very much putting my spin on it) learning a (first?) language as a human is probably a matter of figuring out how to apply existing "machinery" that imposes some of its own structure rather than operating entirely from the information provided to the senses. I'm not saying it's necessarily correct, but it does seem well formed, and LLMs only undermine it in a very weak sense (so far?). To defend the conclusion itself I would have to know a whole lot more about how he estimated information content (needed and available).