3 ms·

I don't get that. To truly predict the next token you have to understand everything in the universe. Maybe people haven't thought enough about what a powerful l

I don't get that. To truly predict the next token you have to understand everything in the universe. Maybe people haven't thought enough about what a powerful learning objective that is. The fact that its simply defined is deceptive because as a heuristic people associate complex definitions with powerful tools.

krapp 3y ago
>To truly predict the next token you have to understand everything in the universe. What do "truly" and "understand" mean here, when LLMs are known for making errors and generating semantically correct but contextually meaningless results? Do they truly understand everything in the universe, or is the appearance of understanding the result of that universe being composed of data created by humans, so that relevant and meaningful results are statistically more likely than not?