3 ms·
> Turn down the temperature and top_p if you don't want the creativity. This also reduces accuracy in real terms. The randomness is used to jump out of local m
by th0ma5 1y ago
> Turn down the temperature and top_p if you don't want the creativity.
This also reduces accuracy in real terms. The randomness is used to jump out of local minima.
- scosman 1y agoThat's at training time, not inference time. And temp/top_p aren't used to escape local minima, methods like SDG batch sampling, Adam, dropout, LR decay, and other techniques do that.
- th0ma5 11mo agoAhh okay, so you really can't escape the indeterminacy?
- scosman 11mo agoYou can zero out temperature and get determinism at inference time. Which is separate from training time where you need forms of randomness to learn. The point is for the quote "all LLMs that I know of rely on entropy and randomness to emulate human creativity" is a runtime parameter you can tweak down to zero, not a fundamental property of the technology.
- th0ma5 11mo agoRight, but my point is is that even if you turn the temperature all the way down, you're not guaranteed to get an accurate or truthful result even though you may get a mostly repeatable deterministic result, and there is still some indeterminacy.