3 ms·
Technically they are designed to solve next-word prediction. It just turns out that good next-word prediction requires understanding the world. (And you can e
by IshKebab 3y ago
Technically they are designed to solve next-word prediction.
It just turns out that good next-word prediction requires understanding the world.
(And you can equally view humans as "next word predictors" anyway.)
- xcv123 3y agoYes that's exactly right. NLP problem solving abilities are a consequence of the fact that in order to predict the next token it actually needs to "understand" all of the previous tokens and how they relate to the next predicted token, and determine that the new output is "logically" coherent in context.