3 ms·
> It’s well-known at this point that GPT-4/GPT-3.5-turbo is non-deterministic, even at temperature=0.0 Interestingly, on another discussion there was a claim t
by cratermoon 3y ago
> It’s well-known at this point that GPT-4/GPT-3.5-turbo is non-deterministic, even at temperature=0.0
Interestingly, on another discussion there was a claim that setting the temperature to 0.0 made gpt-4 deterministic: https://news.ycombinator.com/item?id=36503146 https://news.ycombinator.com/item?id=36503146
- moonchrome 3y agoThis guy probably never did anything nontrivial with the API - you notice almost instantly that the chat models (both 3.5 and 4) are nondeterministic at 0 temperature. Source - built a documentation search bot and had it crap out on me on copy pasted prompts when I was demoing it.
- keskival 3y agoIf you want to make it deterministic, just cache the responses keyed by queries.
- cratermoon 3y agoApparently, and I haven't tested this, just from what I read, the simpler GPT-2 models are deterministic at 0 temperature.