3 ms·
But they can get also real answers even if you have the temperature setting as 0, where it will always pick the highest scoring token?
by mewpmewp2 2y ago
But they can get also real answers even if you have the temperature setting as 0, where it will always pick the highest scoring token?
- buneskamin 2y agoI think he's saying if you set temp to 0 and answers become deterministic, it will appear that the model is just memorising and reciting. The randomness is a hack that 'forces' the model to generalise by deliberately knocking it off the track of the most probable next token