3 ms·
How do you know? its literally non-deterministic.
by tekno45 1y ago
How do you know? its literally non-deterministic.
- r3trohack3r 1y agoMost (all?) AI models I work with are literally deterministic. If you give it the same exact input, you get the same exact output every single time. What most people call “non-deterministic” in AI is that one of those inputs is a _seed_ that is sourced from a PRNG because getting a different answer every time is considered a feature for most use cases. Edit: I’m trying to imagine how you could get a non-deterministic AI and I’m struggling because the entire thing is built on a series of deterministic steps. The only way you can make it look non-deterministic is to hide part of the input from the user.
- throwaway31131 1y ago> I’m trying to imagine how you could get a non-deterministic AI Depends on the machine that implements the algorithm. For example, it’s possible to make ALUs such that 1+1=2 most of the time, but not all the time. … Just ask Intel. (Sorry, I couldn’t resist)
- tekno45 1y agoSo by default. Its non-deterministic for all non power users.
- sumeno 1y agoThis is an incredibly pedantic argument. The common interfaces for LLMs set their temperature value to non-zero, so they are effectively non-deterministic.
- majormajor 1y agoEven after temperature=0 I believe there is some non-determinism at the chip level, similar to https://stackoverflow.com/questions/50744565/how-to-handle-non-determinism-when-training-on-a-gpu https://stackoverflow.com/questions/50744565/how-to-handle-n...
- noosphr 1y agoFrom the good old days: https://152334h.github.io/blog/non-determinism-in-gpt-4/ https://152334h.github.io/blog/non-determinism-in-gpt-4/ (that's been a short two years). Unless something has fundamentally changed since then (which I've not heard about) all sparse models are only deterministic at the batch level, rather than the sample level.