4 ms·

I feel like I'm fighting a losing battle but I don't see why so many people consider LLMs innately non-deterministic, an LLM running on a CPU with greedy decodi

I feel like I'm fighting a losing battle but I don't see why so many people consider LLMs innately non-deterministic, an LLM running on a CPU with greedy decoding would be 100% deterministic and still have the intelligence we associate with LLMs

antonvs 2mo ago
Valid point, it’s why I included the word “traditional”, to try to qualify that. What I meant is something more like explicitly programmed vs. learned. Intelligence can result from learned behavior, but not from explicit programming of rules by humans. An aspect of this is that “learning” is unpredictable - we can’t predict in advance exactly how the resulting model will behave, except broadly. It seems non-deterministic if only by virtue of its complexity, which is beyond anything we’re able to predictively model.
- AdieuToLogic 2mo ago
  > Intelligence can result from learned behavior, but not from explicit programming of rules by humans. This is incorrect. Simulated intelligence can and has been encoded explicitly by humans defining rules programmatically in the form of expert systems[0]. 0 - https://en.wikipedia.org/wiki/Expert_system https://en.wikipedia.org/wiki/Expert_system
  - antonvs 2mo ago
    I've worked on expert systems. I don't agree that they achieve "simulated intelligence". They're preprogrammed with a set of domain-specific rules that are trivially simple by comparison to even relatively simple and small neural networks.
    - AdieuToLogic 2mo ago
      > I've worked on expert systems. I also have worked on/with expert systems. > I don't agree that they achieve "simulated intelligence". They're preprogrammed with a set of domain-specific rules that are trivially simple by comparison to even relatively simple and small neural networks. This position does not account for fuzzy logic[0] nor an expert system's ability to produce an answer of "I do not know and here is why", which neural networks are incapable of doing. I am not saying expert systems are "better" than ANNs as both are algorithms having significant value for what they provide. What I am saying is neural networks are pattern-matching algorithms, quite useful in their own right, and do not possess the ability to identify the lack of existence. 0 - https://en.wikipedia.org/wiki/Fuzzy_logic https://en.wikipedia.org/wiki/Fuzzy_logic
      - antonvs 2mo ago
        No, fuzzy logic doesn't change what I said at all. In terms of intelligence, both of those technologies were at best limited and simplistic attempts at achieving what LLMs have actually achieved. Comparing the two in 2026 seems like a bit of a joke to me. I'm not saying there's no role in future for traditional expert systems or fuzzy logic (or hand-written code, for that matter), but to claim they're "intelligence" or even "simulated intelligence" implies such a trivial definition of "intelligence" as to make it a useless term. > an expert system's ability to produce an answer of "I do not know and here is why", which neural networks are incapable of doing. Why do you believe that? Here's an excerpt from a response I received from Claude tonight: > "I want to be honest about a limitation: I can't reliably confirm fine construction details — like exactly which sub-assembly is bolted to the spoke flange versus the fixed axle — from a marketing cutaway graphic at typical web resolution. Those images tend to be stylized/exploded-view illustrations meant to show 'there's a battery and a motor in here,' not engineering-accurate cross-sections with clear rotating/stationary boundaries marked." This is after it examined two images I provided it with, and related it to the discussion we'd been having. This demonstrates that it can indeed answer "I do not know and here is why", so your idea about what neural networks "are incapable of doing" is clearly incorrect. And even if I grant your trivial threshold for intelligence, an interaction like that one clearly demonstrates a far superior degree of multi-modal intelligence, reasoning, and understanding that no expert system or fuzzy logic has ever even come close to achieving.
AdieuToLogic 2mo ago
> I feel like I'm fighting a losing battle but I don't see why so many people consider LLMs innately non-deterministic ... Because LLMs are artificial neural networks[0] (ANN), which are statistical in nature, and thus intrinsically non-deterministic. Pretty much every AI algorithm has randomness involved in its definition and many (most?) incorporate probabilities. 0 - https://en.wikipedia.org/wiki/Neural_network_(machine_learning) https://en.wikipedia.org/wiki/Neural_network_(machine_learni...
- ChadNauseam 2mo ago
  Now that you mention it, I think "statistical" might be a good word choice as it probably closely matches what people mean when they say an LLM is nondeterministic. However, on a technical level, neural network inference truly is inherently deterministic. The only nondeterminism in LLMs is parallelism-dependency in the way floating point operations are carried out and in the decoding strategy
  - vrighter 2mo ago
    only if you bias your "random sampling of the distributions it gives" Fixing the seed is still intentional bias. Or you could force it to always take the one token with the highest probability, but that is still biased sampling. Deterministic, sure, but intentionally wrong just to avoid a technically
    - ChadNauseam 2mo ago
      There's nothing inherently biased or intentionally wrong with greedy decoding. Why would there be? Let's say you're trying to predict what an expert doctor would say to a patient with cancer. You think there's a 99% chance the doctor would say "start chemotherapy" and a 1% chance the doctor would say "don't worry about it". If you have to pick one to output, you'll pick "start chemotherapy". It would be crazy to say the best thing to do is to roll a 100-sided die to determine your answer. A language model is literally in that exact situation.
  - AdieuToLogic 2mo ago
Terr_ 2mo ago
I've seen this debate several times and often there's a terminology issue, where the same word isn't being interpreted the same way by different sides. Often it's a difference between repeatable versus predictable, or whether a system has chaotic aspects like the configurations of a double-pendulum or weather-forecasting. Sometimes it's the difference between determinism in-theory versus in-practice, especially when various optimizations are being applied to save money.
- yencabulator 2mo ago
  Also, people seem to not realize that chaos can be 100% deterministic. Consider Mandelbrot or Conway's Game of Life. The LLM inference process can be 100% deterministic but the weights can still make the end result quite chaotic. Just because temperature>0 improves results doesn't mean its an innate part of the mechanism. Just because scale-out architectures introduce jitter in communication doesn't mean that's an innate part of the mechanism.
vrighter 2mo ago
deterministic but completely unpredictable is not really any more useful

antonvs 2mo ago

Valid point, it’s why I included the word “traditional”, to try to qualify that. What I meant is something more like explicitly programmed vs. learned. Intelligence can result from learned behavior, but not from explicit programming of rules by humans. An aspect of this is that “learning” is unpredictable - we can’t predict in advance exactly how the resulting model will behave, except broadly. It seems non-deterministic if only by virtue of its complexity, which is beyond anything we’re able to predictively model.

AdieuToLogic 2mo ago
> Intelligence can result from learned behavior, but not from explicit programming of rules by humans. This is incorrect. Simulated intelligence can and has been encoded explicitly by humans defining rules programmatically in the form of expert systems[0]. 0 - https://en.wikipedia.org/wiki/Expert_system https://en.wikipedia.org/wiki/Expert_system
- antonvs 2mo ago
  I've worked on expert systems. I don't agree that they achieve "simulated intelligence". They're preprogrammed with a set of domain-specific rules that are trivially simple by comparison to even relatively simple and small neural networks.
  - AdieuToLogic 2mo ago
    > I've worked on expert systems. I also have worked on/with expert systems. > I don't agree that they achieve "simulated intelligence". They're preprogrammed with a set of domain-specific rules that are trivially simple by comparison to even relatively simple and small neural networks. This position does not account for fuzzy logic[0] nor an expert system's ability to produce an answer of "I do not know and here is why", which neural networks are incapable of doing. I am not saying expert systems are "better" than ANNs as both are algorithms having significant value for what they provide. What I am saying is neural networks are pattern-matching algorithms, quite useful in their own right, and do not possess the ability to identify the lack of existence. 0 - https://en.wikipedia.org/wiki/Fuzzy_logic https://en.wikipedia.org/wiki/Fuzzy_logic
    - antonvs 2mo ago
      No, fuzzy logic doesn't change what I said at all. In terms of intelligence, both of those technologies were at best limited and simplistic attempts at achieving what LLMs have actually achieved. Comparing the two in 2026 seems like a bit of a joke to me. I'm not saying there's no role in future for traditional expert systems or fuzzy logic (or hand-written code, for that matter), but to claim they're "intelligence" or even "simulated intelligence" implies such a trivial definition of "intelligence" as to make it a useless term. > an expert system's ability to produce an answer of "I do not know and here is why", which neural networks are incapable of doing. Why do you believe that? Here's an excerpt from a response I received from Claude tonight: > "I want to be honest about a limitation: I can't reliably confirm fine construction details — like exactly which sub-assembly is bolted to the spoke flange versus the fixed axle — from a marketing cutaway graphic at typical web resolution. Those images tend to be stylized/exploded-view illustrations meant to show 'there's a battery and a motor in here,' not engineering-accurate cross-sections with clear rotating/stationary boundaries marked." This is after it examined two images I provided it with, and related it to the discussion we'd been having. This demonstrates that it can indeed answer "I do not know and here is why", so your idea about what neural networks "are incapable of doing" is clearly incorrect. And even if I grant your trivial threshold for intelligence, an interaction like that one clearly demonstrates a far superior degree of multi-modal intelligence, reasoning, and understanding that no expert system or fuzzy logic has ever even come close to achieving.

AdieuToLogic 2mo ago

> I feel like I'm fighting a losing battle but I don't see why so many people consider LLMs innately non-deterministic ... Because LLMs are artificial neural networks[0] (ANN), which are statistical in nature, and thus intrinsically non-deterministic. Pretty much every AI algorithm has randomness involved in its definition and many (most?) incorporate probabilities. 0 - https://en.wikipedia.org/wiki/Neural_network_(machine_learning) https://en.wikipedia.org/wiki/Neural_network_(machine_learni...

ChadNauseam 2mo ago
Now that you mention it, I think "statistical" might be a good word choice as it probably closely matches what people mean when they say an LLM is nondeterministic. However, on a technical level, neural network inference truly is inherently deterministic. The only nondeterminism in LLMs is parallelism-dependency in the way floating point operations are carried out and in the decoding strategy
- vrighter 2mo ago
  only if you bias your "random sampling of the distributions it gives" Fixing the seed is still intentional bias. Or you could force it to always take the one token with the highest probability, but that is still biased sampling. Deterministic, sure, but intentionally wrong just to avoid a technically
  - ChadNauseam 2mo ago
    There's nothing inherently biased or intentionally wrong with greedy decoding. Why would there be? Let's say you're trying to predict what an expert doctor would say to a patient with cancer. You think there's a 99% chance the doctor would say "start chemotherapy" and a 1% chance the doctor would say "don't worry about it". If you have to pick one to output, you'll pick "start chemotherapy". It would be crazy to say the best thing to do is to roll a 100-sided die to determine your answer. A language model is literally in that exact situation.
- AdieuToLogic 2mo ago

Terr_ 2mo ago

I've seen this debate several times and often there's a terminology issue, where the same word isn't being interpreted the same way by different sides. Often it's a difference between repeatable versus predictable, or whether a system has chaotic aspects like the configurations of a double-pendulum or weather-forecasting. Sometimes it's the difference between determinism in-theory versus in-practice, especially when various optimizations are being applied to save money.

yencabulator 2mo ago
Also, people seem to not realize that chaos can be 100% deterministic. Consider Mandelbrot or Conway's Game of Life. The LLM inference process can be 100% deterministic but the weights can still make the end result quite chaotic. Just because temperature>0 improves results doesn't mean its an innate part of the mechanism. Just because scale-out architectures introduce jitter in communication doesn't mean that's an innate part of the mechanism.

vrighter 2mo ago

deterministic but completely unpredictable is not really any more useful