2 ms·
"No they are not copied off of human behavior in any way shape or fashion." The pre-training phase produces the next-token predictors. The post-training phase
by simonw 1y ago
"No they are not copied off of human behavior in any way shape or fashion."
The pre-training phase produces the next-token predictors. The post-training phase is where its shown examples of selected human behavior for it to imitate - examples of conversation patterns, expert code production, how to argue a point... there's an enormous amount of "copying human behavior" involved in producing a useful LLM.
- ACCount37 1y agoWhy single out SFT? It's not like the pre-training dataset didn't contain any examples of human behaviors for an LLM to copy. SFT is just a more selective process. And a lot of how it does what it does is less "teach this LLM new tricks" and more "teach this LLM how to reach into its bag of tricks and produce the right tricks at the right times".
- simonw 1y agoI think it is a more clear example of deliberately teaching a model specific ways to behave based on human examples.
- somenameforme 1y agoI think what he's saying (and what I would at least) is that again all you're doing is the exact same thing - tuning the weights that drive the correlations. For an analog, in a video game if you code a dragon such that its elevation changes over time while you play a wing flapping animation, you're obviously not teaching it dragon-like behaviors, but rather simply trying to create a mimicry of the appearance of flying using relatively simple mathematical tools and 'tricks.' And indeed even basic neural network game bots benefit from RLHF/SFT.