4 ms·

Traditional non-chaotic systems* LLMs are deterministic. They are chaotic, which people confuse for non-deterministic.

by Zambyte 10d ago

sp527 10d ago
> They are chaotic, which people confuse for non-deterministic. That's an odd argument, because a lot of people who have struggled to decipher complex chaotic systems would tell you this is a distinction without much of a difference.
- gmm1990 10d ago
  Yeah if you give the llm the exact same input over and over you really shouldn’t be using an llm.
- Zambyte 10d ago
  It's at least important from philosophical perspective regarding whether LLMs have free will or not. It's hard to say whether or not a human thought or action is driven by free will, people have debated it for centuries. We can provide the same input to an LLM and get the same output, because they are deterministic. Surely it is obvious that a pure function does not have free will, no matter how expensive it is to compute.
SkyBelow 10d ago
I used that description some months ago and I think you are the first person I see who put it the same way. The only gotcha with this is that they are theoretically deterministic, but rarely in practice. A few examples: - Harness specific settings that user can't control (anything from timestamp to prng seeding. - Batching requests in a way that leads to a single request being processed different depending upon the batch (say MoE where your first choice expert is assigned to someone else's token so you go to your second choice vs a batch where you get your first choice). - Graphics card itself carrying out floating point arithmetic in slightly different orders leading to floating point non associativity causing different outputs. But all of these can be controlled for (at some cost) and the model can be ran deterministically. For the average user, it might as well be non-deterministic, but when considering theoretical capabilities, chaotic deterministic system seems the better description.
bluegatty 9d ago
I see your point but, from a systems perspective they are pragmatically non-deterministic. The use of LLM in any system creates non-determinism in any practical sense.

Traditional non-chaotic systems* LLMs are deterministic. They are chaotic, which people confuse for non-deterministic.

by Zambyte 10d ago

Traditional non-chaotic systems* LLMs are deterministic. They are chaotic, which people confuse for non-deterministic.

sp527 10d ago

> They are chaotic, which people confuse for non-deterministic. That's an odd argument, because a lot of people who have struggled to decipher complex chaotic systems would tell you this is a distinction without much of a difference.

gmm1990 10d ago
Yeah if you give the llm the exact same input over and over you really shouldn’t be using an llm.
Zambyte 10d ago
It's at least important from philosophical perspective regarding whether LLMs have free will or not. It's hard to say whether or not a human thought or action is driven by free will, people have debated it for centuries. We can provide the same input to an LLM and get the same output, because they are deterministic. Surely it is obvious that a pure function does not have free will, no matter how expensive it is to compute.

SkyBelow 10d ago

I used that description some months ago and I think you are the first person I see who put it the same way. The only gotcha with this is that they are theoretically deterministic, but rarely in practice. A few examples: - Harness specific settings that user can't control (anything from timestamp to prng seeding. - Batching requests in a way that leads to a single request being processed different depending upon the batch (say MoE where your first choice expert is assigned to someone else's token so you go to your second choice vs a batch where you get your first choice). - Graphics card itself carrying out floating point arithmetic in slightly different orders leading to floating point non associativity causing different outputs. But all of these can be controlled for (at some cost) and the model can be ran deterministically. For the average user, it might as well be non-deterministic, but when considering theoretical capabilities, chaotic deterministic system seems the better description.

bluegatty 9d ago

I see your point but, from a systems perspective they are pragmatically non-deterministic. The use of LLM in any system creates non-determinism in any practical sense.