2 ms·

> The only nondeterminism in LLMs is parallelism-dependency in the way floating point operations are carried out and in the decoding strategy This assertion is

> The only nondeterminism in LLMs is parallelism-dependency in the way floating point operations are carried out and in the decoding strategy This assertion is "oddly" similar to the GPT answer "neural network inference determinism" produced: Neural network inference is often non-deterministic due to factors like floating-point arithmetic and concurrent execution, which can lead to variations in output even with the same input. Surely this is but a coincidence. Regarding your previous statement: > However, on a technical level, neural network inference truly is inherently deterministic. This holds for a vanishingly small set of conditions, none of which include randomness, nor when context and transformers are involved, let alone underlying model evolution (thus making model use over time non-deterministic).

ChadNauseam 1mo ago
> This assertion is "oddly" similar to the GPT answer "neural network inference determinism" produced My statement and GPT's statement are both correct answers to the same question, so I think it makes sense that they would be similar. Are you accusing me of having paraphrased an LLM in writing my answer? I did not, I just remembered having read Thinky's post on the subject [0], which GPT has probably read also. > This holds for a vanishingly small set of conditions, none of which include randomness, nor when context and transformers are involved, let alone underlying model evolution (thus making model use over time non-deterministic). There are plenty of ways to introduce nondeterminism into any system. By your standards, I doubt you could point to a single deterministic system in the world. print("hello, world") is only deterministic if your CPU is properly shielded from cosmic rays and your OS isn't out of memory etc. There are some inherently nondeterministic processes, like the stochastic methods used to train models or the random sampling used at inference time if you have temperature!=0, but inference under greedy decoding is conceptually deterministic. [0]: https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/ https://thinkingmachines.ai/blog/defeating-nondeterminism-in...

ChadNauseam 1mo ago

> This assertion is "oddly" similar to the GPT answer "neural network inference determinism" produced My statement and GPT's statement are both correct answers to the same question, so I think it makes sense that they would be similar. Are you accusing me of having paraphrased an LLM in writing my answer? I did not, I just remembered having read Thinky's post on the subject [0], which GPT has probably read also. > This holds for a vanishingly small set of conditions, none of which include randomness, nor when context and transformers are involved, let alone underlying model evolution (thus making model use over time non-deterministic). There are plenty of ways to introduce nondeterminism into any system. By your standards, I doubt you could point to a single deterministic system in the world. print("hello, world") is only deterministic if your CPU is properly shielded from cosmic rays and your OS isn't out of memory etc. There are some inherently nondeterministic processes, like the stochastic methods used to train models or the random sampling used at inference time if you have temperature!=0, but inference under greedy decoding is conceptually deterministic. [0]: https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/ https://thinkingmachines.ai/blog/defeating-nondeterminism-in...