> And of course, they have zero understanding of anything at all. They’re a next-token-predicting machine with an unimaginably large training set. Oof, this fa

> And of course, they have zero understanding of anything at all. They’re a next-token-predicting machine with an unimaginably large training set. Oof, this fallacy is so frustrating. Being a next-token-predicting machine doesn't imply "having no understanding of anything". The best way to correctly predict the next token in the answer to a question is to understand the question properly, and reason your way to that true answer. So that's what it tries to do (with inconsistent success). Seeing the tool this way should also remove the disbelief that it works. It works because it starts to model not only the surface statistics, but the causes of those statistics, in order to make predictions that are less wrong. That's how it gets a world model from a bunch of sample text.