3 ms·
> LLM outputs are nondeterministic. LLM outputs are deterministic. There is no intrinsic source of randomness. Users can add randomness (temperature) to the ou
by csomar 1y ago
> LLM outputs are nondeterministic.
LLM outputs are deterministic. There is no intrinsic source of randomness. Users can add randomness (temperature) to the output and modify it.
> But this article is based on a model of LLM code generation from 6 months ago
There hasn't been much change in models from 6 months ago. What happened is that we have better tooling to sift through the randomly generated outputs.
I don't disagree with your message. You are being downvoted because a lot of software developers are butt-hurt by it. It is going to force a change in the labor market for developers. In the same way the author is butt-hurt that they didn't buy Bitcoin in the very early days (as they were aware of it) and missed the boat on that.
- tptacek 1y agoThere hasn't been much change in models from 6 months ago. I made the same claim in a widely-circulated piece a month or so back, and have come to believe it was wildly false, the dumbest thing I said in that piece.
- csomar 1y agoI have my own test to measure performance: https://omarabid.com/gpt3-now https://omarabid.com/gpt3-now So far the only model that showed significant advancement and differentiation was GPT-4.5. I advise to look at the problem and read GPT-4.5 answer. It'll show the difference to other "normal models" (including GPT-3.5) as it shows considerable levels of understanding. Other normal models are now more chatty and have a bit more data. But they do not show increased intelligence.
- Karrot_Kream 1y agoI was able to have Opus 4 one-shot it. Happy to share a screenshot if that wasn't your experience.
- csomar 1y agoInterested to see your Opus 4 one-shot. I tried it very recently on Opus 4 and it burbled non-sense.
- Karrot_Kream 1y agoSorry for the delay, I'm out for the weekend I'll hey you it tomorrow!
- reasonableklout 1y agoNit: in practice, even at temperature 0, production LLM implementations have some non-determinism. One reason is because many floating point computations are technically non-commutative even when the mathematical operation is, and the order can vary if they are carried out in parallel by the GPU. For example, see: https://www.twosigma.com/articles/a-workaround-for-non-determinism-in-tensorflow/ https://www.twosigma.com/articles/a-workaround-for-non-deter...
- jkhdigital 1y agoI ran into this a bit while working on my PhD research that used LLMs for steganography. The output had to be deterministic to reverse the encoding, and it was—as long as you used the same hardware. Encoding a message on a PC and then trying to decode on a phone broke everything.