"predicting the next token well means that you understand the underlying reality that led to the creation of that token" People on the AI-hype side of things t

"predicting the next token well means that you understand the underlying reality that led to the creation of that token" People on the AI-hype side of things tend to believe this, but I really fundamentally don't. It's become a philosophical debate at this point (what does it mean to "understand" something, etc.)