3 ms·
Im saying temperature is a hack to make the models actually produce real answers.
by ActorNightly 2y ago
Im saying temperature is a hack to make the models actually produce real answers.
- mewpmewp2 2y agoBut they can get also real answers even if you have the temperature setting as 0, where it will always pick the highest scoring token?
- buneskamin 2y agoI think he's saying if you set temp to 0 and answers become deterministic, it will appear that the model is just memorising and reciting. The randomness is a hack that 'forces' the model to generalise by deliberately knocking it off the track of the most probable next token