2 ms·
They are next token producers. I would only call it a predictor, if it's trained to predict tokens (ie just after pretraining). Just like humans produce one w
by eru 8d ago
They are next token producers. I would only call it a predictor, if it's trained to predict tokens (ie just after pretraining).
Just like humans produce one word after another when they talk, but they don't generally try to imitate other humans.
- wood_spirit 8d agoDon’t people pick up language, vocabulary and dialect from those around them? Perhaps it’s subconscious but humans are imitating other humans all the time?
- eru 8d agoIt's a mix. Yes, you imitate how others speak, but when you are trying to solve a problem, you don't try to predict how others would complete a text. (Well, unless you follow 'what token would Jesus pick?' / 'what would Jesus do'.)