> empty transformer models will not do this. Do we have evidence for that? Of course those kids did have a real world to communicate about, and that creates th

> empty transformer models will not do this. Do we have evidence for that? Of course those kids did have a real world to communicate about, and that creates the need for a language. I think if you had AI models that had a world to interact with and other models to communicate with and learn from, they might be able to develop their own language too. But this does get to another of Chomsky's points: the need to have a world model. To current LLMs, the only world model they have is text they read on the internet. And they only communicate with us, and I don't think they learn from their communication with us. So that's unavoidably limiting them. But I think it would be a very interesting experiment to give them a world to interact with and other similar models to communicate with, to see if they develop some sort of language (which would be completely incomprehensible to us of course; that's another hurdle). I agree that LLMs don't disprove his inborn grammar argument, but they don't support it either. I think more research is needed.