3 ms·

it annoys me how eager people are to hurl the word stochastic as pejorative. Statistics are a great tool for gleaning information from stochastic processes; sta

by fsckboy 3mo ago

it annoys me how eager people are to hurl the word stochastic as pejorative. Statistics are a great tool for gleaning information from stochastic processes; statistics don't contribute randomness. Random sampling is necessary in order not to bias a sample, it's not used to contribute randomness to the sample but to preserve/measure the underlying distribution. (not meant to imply that training is random sampling)

bigstrat2003 3mo ago
It's a pejorative only because determinism is what makes computers useful in the first place. You get a consistent result, every single time, unlike if you have a human in the loop. Because LLMs are stochastic, they have removed the thing that makes computers useful to us, thus it's a pejorative.
- DangitBobby 3mo ago
  It turns out that determinism isn't what makes computers useful in the first place.
- naasking 3mo ago
  1. Determinism is a very small subset of what makes computers useful. Non-determinism like stochasticity is literally everywhere, like random seeds. 2. LLMs are detemrinistic. They have a parameter to tune how stochastic they are.
  - TacticalCoder 3mo ago
    Sorry at first I downvoted you instead of upvoting you.
- TacticalCoder 3mo ago
  What do you mean by determinism here? That you ask the computer 2+2 and it gives 4 as you expected or that if you ask the computer 2+2 and it hallucinates 5, you want it to always hallucinate 5? Which one it doesn't do for you? Does it sometimes answer 4, sometimes 5? There are definitely models that will always give 100% of the time the exact same answer, bit-for-bit, given the same input and seed. There are generative image models you can run locally doing just that. But you can also run some the SOTA chinese LLMs at "temperature 0" and, given the same input, they'll always give you the exact same output. Because it's just a machine doing computation. In the beginning of LLMs some "engineers" have tried to hand-wave non-sensical explanation as to why LLMs couldn't possibly be deterministic but: the open-weights models that can be run in a 100% deterministic way are way more powerful than the SOTA models of back then, so those explanation were pure rubbish bollocks. Now of course if you run a complex chain of events, with LLMs doing calls to other LLMs, where some of them go fetch infos on unreliable networks, with infos that may have changed, then, logically, you won't always get the same answer.

by fsckboy 3mo ago

bigstrat2003 3mo ago

It's a pejorative only because determinism is what makes computers useful in the first place. You get a consistent result, every single time, unlike if you have a human in the loop. Because LLMs are stochastic, they have removed the thing that makes computers useful to us, thus it's a pejorative.

DangitBobby 3mo ago
It turns out that determinism isn't what makes computers useful in the first place.
naasking 3mo ago
1. Determinism is a very small subset of what makes computers useful. Non-determinism like stochasticity is literally everywhere, like random seeds. 2. LLMs are detemrinistic. They have a parameter to tune how stochastic they are.
- TacticalCoder 3mo ago
  Sorry at first I downvoted you instead of upvoting you.
TacticalCoder 3mo ago
What do you mean by determinism here? That you ask the computer 2+2 and it gives 4 as you expected or that if you ask the computer 2+2 and it hallucinates 5, you want it to always hallucinate 5? Which one it doesn't do for you? Does it sometimes answer 4, sometimes 5? There are definitely models that will always give 100% of the time the exact same answer, bit-for-bit, given the same input and seed. There are generative image models you can run locally doing just that. But you can also run some the SOTA chinese LLMs at "temperature 0" and, given the same input, they'll always give you the exact same output. Because it's just a machine doing computation. In the beginning of LLMs some "engineers" have tried to hand-wave non-sensical explanation as to why LLMs couldn't possibly be deterministic but: the open-weights models that can be run in a 100% deterministic way are way more powerful than the SOTA models of back then, so those explanation were pure rubbish bollocks. Now of course if you run a complex chain of events, with LLMs doing calls to other LLMs, where some of them go fetch infos on unreliable networks, with infos that may have changed, then, logically, you won't always get the same answer.