I'd just like to say, with unfortunately little to add, that your comments on this article are terrific to read. You've captured perfectly how I have felt about

I'd just like to say, with unfortunately little to add, that your comments on this article are terrific to read. You've captured perfectly how I have felt about LLMs roughly from the first time they came out to how I still feel now. They're utterly amazingly technology that truly do feel like magic, except that I have enough of a background in math modeling and ML to de-mystify them. But the key difference between a model and a human is exactly what you just said. It's what animals can do on the margin. Nobody taught humans language. Each of us individually who are alive today, sure. But go back far enough and humanity invented language. We directly interact with the physical world, develop mental models of it, observe that we are able to make sounds and symbols and somehow come to a mutual agreement that they should mean something in rough analogy to these independent but sufficiently similar mental models. That is magic. Nobody, no programmer, no mathematician, no investor, has any idea how humanity did that, and has no idea how to get a machine to do it, either. Replicating the accomplishments of something else is a tremendous feat and it will get our software very, very far, maybe as far as we ever need to really get it. But it is not doing what animals did. It didn't just figure this shit out on its own. Maybe somewhat ironically, I don't even know that this is a real limitation that current techniques for developing statistical models can't overcome. Put some "AIs" loose in robot bodies, let them freely move about the world trying to accomplish the simple goal of continuing to exist, with cooperation allowed, and they may very well develop ways to encode knowledge, share it with each other, and write it down somehow to pass on to the future so they don't need to continually re-learn everything, especially if they get millions of years to do it. It's obvious, though, that we don't even want this. It might be interesting purely as an experiment, but it probably isn't going to lead to any useful tools. What we do now actually does lead to useful tools. To me, that should tell us something in these discussions. Trying to figure if X piece of software is or isn't cognitively equal to or better than a human in some respect is a tiring, pointless exercise. Who cares? Is it useful to us or not? What are its uses? What are its limitations? We're just trying to automate some toil here, aren't we? We're not trying to play God and create a separate form of life with its own purposes.