3 ms·
> But there's a lot of re-inforcement learning afterwards. That just shifts the distribution of tokens produced. Ultimately they are still just next token pred
by disgruntledphd2 8d ago
> But there's a lot of re-inforcement learning afterwards.
That just shifts the distribution of tokens produced. Ultimately they are still just next token predictors.
Like, even "reasoning" models basically work by generating more tokens at inference time, and using them to shift the distribution towards more useful outcomes (in some cases).
- eru 8d agoThey are next token producers. I would only call it a predictor, if it's trained to predict tokens (ie just after pretraining). Just like humans produce one word after another when they talk, but they don't generally try to imitate other humans.
- wood_spirit 8d agoDon’t people pick up language, vocabulary and dialect from those around them? Perhaps it’s subconscious but humans are imitating other humans all the time?
- eru 8d agoIt's a mix. Yes, you imitate how others speak, but when you are trying to solve a problem, you don't try to predict how others would complete a text. (Well, unless you follow 'what token would Jesus pick?' / 'what would Jesus do'.)