4 ms·
This guy probably never did anything nontrivial with the API - you notice almost instantly that the chat models (both 3.5 and 4) are nondeterministic at 0 tempe
by moonchrome 3y ago
This guy probably never did anything nontrivial with the API - you notice almost instantly that the chat models (both 3.5 and 4) are nondeterministic at 0 temperature. Source - built a documentation search bot and had it crap out on me on copy pasted prompts when I was demoing it.
- keskival 3y agoIf you want to make it deterministic, just cache the responses keyed by queries.
- cratermoon 3y agoApparently, and I haven't tested this, just from what I read, the simpler GPT-2 models are deterministic at 0 temperature.