5 ms·
An LLM like ChatGPT ends up with a bunch of probabilities for the next word-ish. A random number generator then picks from them, weighted by probability, and th
by danielvf 4y ago
An LLM like ChatGPT ends up with a bunch of probabilities for the next word-ish. A random number generator then picks from them, weighted by probability, and then the process loops. So while the weights are static, the actual execution is very much non-deterministic.
- amalcon 4y agoWhile I can't say whether ChatGPT or others are actually doing this, it's entirely possible to use a PRNG in that spot with no loss of effectiveness. PRNGs are specifically designed to create the appearance of randomness out of a process that is in fact wholly deterministic.
- gpderetta 4y agoDo they use an actual physical source of randomness for every random number? Or is it a prng? Because in that case it is fully deterministic. And consider the case that in fact they could use actual random numbers. Would it make a difference?
- plg94 4y agoJust because we cannot determine something doesn't mean it's non-deterministic. The question of the existence of free will is very similar – if not equivalent – to the existence of real randomness. I.e. a dice roll may be deterministic if you just know the right set of complicated equations for gravity, air resistance, turbulence etc.
- v64 4y ago> the actual execution is very much non-deterministic. Only as a default. Temperature can be turned to 0 and other token selection settings changed from the defaults to create deterministic output from a given prompt.