3 ms·
Well, yes and no. By non-deterministic I think people really mean "chaotic" in the chaos theory sense. Small perturbations in the input lead to wild and unpred
by gpm 2mo ago
Well, yes and no.
By non-deterministic I think people really mean "chaotic" in the chaos theory sense. Small perturbations in the input lead to wild and unpredictable changes in the output. Even with temperature parameters a fixed PRNG seed could mean an LLM was just chaotic and not technically non-deterministic.
But more literally while LLMs are in theory deterministic (though perhaps not inference providers implementations if there's anything like a race condition affecting how things are rounded when added together) - we use the LLMs in harnesses that aren't. There are very likely races in the terminal outputs, dates both intentionally put in the context and accidentally leaked to the context, things like that.
- gnunez 2mo agoOk. I see. I guess people are not referring to the raw models themselves when they say non-deterministic, but are also including the harness used in conjunction with the model. Then, in that case, for the exact same input you could get a non-deterministic output. But the model itself and all the mathematical machinery around the model is still very much deterministic. I guess if we really needed to, we could construct a deterministic agent harness. But in most use cases we probably want some chaotic behavior to increase our chances of stumbling on the desired results. Thank you for the clarification