10 ms·
Why are they non deterministic? Is randomness intentionally injected? Or intrinsic to the approach?
by richk449 4y ago
Why are they non deterministic? Is randomness intentionally injected? Or intrinsic to the approach?
- williamstein 4y agoCheck out the temperature docs in the API reference manual: https://platform.openai.com/docs/api-reference/completions/create#completions/create-temperature https://platform.openai.com/docs/api-reference/completions/c...
- alfiopuglisi 4y agoActually, LLMs are completely deterministic. Their output is a list of possible word, ordered by probability. If you always choose the highest-ranked word for the next iteration, they will always generate the same sequence for the same prompt. In all current implementations, a small amount of randomness (called "temperature") is added. In practice, the higher the temperature, the more probable is that a word down the list is selected, rather than the top one.