4 ms·
A langugage model sees a pile of examples with digits and imitates those examples. A reasoning model sees the inner principle behind this pile, and instead of i
by akomtu 3y ago
A langugage model sees a pile of examples with digits and imitates those examples. A reasoning model sees the inner principle behind this pile, and instead of imitating examples, it uses the learnt principle to produce answers.
- antonvs 3y agoHow do you know this? What's an example of a "reasoning model"? If the only example is the human mind, for all we know our reasoning capability and ability to discern principles could work much the same way, and it's just some more subtle differences that lead to the differences in capabilities. There are plenty of cases where it appears as though GPT has discerned the "inner principle" behind something to produce answers.
- niam 3y agoLanguage models aren't really optimized for imitation though, they're optimized to predict. One means of prediction, which models have found to be effective in many contexts (especially when short on training time/compute), is comparable to imitation. But this isn't to say that language models are incapable of establishing "inner principles".