https://www.youtube.com/watch?v=DQuiso493ro&t=3309s https://www.youtube.com/watch?v=DQuiso493ro&t=3309s "Have you seen the paper that's called Modern Language

https://www.youtube.com/watch?v=DQuiso493ro&t=3309s https://www.youtube.com/watch?v=DQuiso493ro&t=3309s "Have you seen the paper that's called Modern Language Models Refute Chomsky's Approach to Language? The one by Steven Piantadosi. If so, what do you make of it? Well, I mean, unlike a lot of people who write about this, he does know about large language models, but the article makes absolutely no sense. It has a minor problem and a major problem. The minor problem is that it's beyond absurdity to think that you can learn anything about a two bunch of supercomputers scanning 50 terabytes of data, looking for statistical regularities, and stringing them together. To think that from that you can learn anything about an infant is so beyond absurdity that it's not worth talking about. That's the minor problem. The major problem is that, in principle, you can learn nothing. In principle, make it 100 terabytes, use 20 supercomputers. Just bring out the, in principle, impossibility. Even worse, for more clearly, for a very simple reason, these systems work just as well for impossible languages that children can't acquire as they do for possible languages that children do acquire. It's kind of as if a physicist came along and said, I've got a great new theory. It accounts for a lot of things that happen, a lot of things that can't possibly happen, and I can't make any distinction among them. We would just laugh. I mean, any explanation of anything says, here's why things happen, here's why other things don't happen. You can't make that distinction. You're doing nothing. That's irremediable. It's built into the nature of the systems. The more sophisticated they become at dealing with actual language, the more it's demonstrated that they're telling us nothing, in principle, about language, about learning, or about cognition. So, there's a minor problem with this paper, and a major one. The minor problem is the simple absurdity of thinking that looking at anything of the scale could tell you about what's happening. The major problem is you can't do it in principle. Yes, I heard you describe this as if a physicist came along and said, I have a theory, and it's two words. Anything goes. Well, that's basically the large language models. You give it a system that's designed in order to violate the principles of language, it'll do just as well. Maybe better, because it can use simple algorithms that aren't used by natural language. So, it's basically, like I said, or like I say, suppose some guy comes along with an improvement on the periodic table, says I got a theory that includes all the possible elements, even those that haven't been discovered, and all kinds of impossible ones, and I can't tell any difference. It's not an improvement on the periodic table. That's telling you nothing about chemistry. That's built into the design of the system. It's not remediable."