4 ms·
It is not accurate to say that an LLM like ChatGPT predicts anything. It is trained to maximize a score function, so it is more like trying to win a game where
by toxik 3y ago
It is not accurate to say that an LLM like ChatGPT predicts anything. It is trained to maximize a score function, so it is more like trying to win a game where the moves are word choices.
- akoboldfrying 3y agoThe game is predicting the next word a person would write.
- logicchains 3y agoNot after RLHF.
- akoboldfrying 3y agoI'd say RLHF bends the game towards predicting what words a "helpful", "respectful" person would write next, for values of "helpful" and "respectful" that vary according to each person involved in scoring (but which are carefully shaped by the people choosing those people, and paying for their time).
- toxik 3y agoThat’s not true and that was the entire point of my comment. It’s like playing chess, you’re not predicting your next best move, you’re searching for it.