3 ms·
Most (all?) AI models I work with are literally deterministic. If you give it the same exact input, you get the same exact output every single time. What most
by r3trohack3r 1y ago
Most (all?) AI models I work with are literally deterministic. If you give it the same exact input, you get the same exact output every single time.
What most people call “non-deterministic” in AI is that one of those inputs is a _seed_ that is sourced from a PRNG because getting a different answer every time is considered a feature for most use cases.
Edit: I’m trying to imagine how you could get a non-deterministic AI and I’m struggling because the entire thing is built on a series of deterministic steps. The only way you can make it look non-deterministic is to hide part of the input from the user.
- throwaway31131 1y ago> I’m trying to imagine how you could get a non-deterministic AI Depends on the machine that implements the algorithm. For example, it’s possible to make ALUs such that 1+1=2 most of the time, but not all the time. … Just ask Intel. (Sorry, I couldn’t resist)
- tekno45 1y agoSo by default. Its non-deterministic for all non power users.
- sumeno 1y agoThis is an incredibly pedantic argument. The common interfaces for LLMs set their temperature value to non-zero, so they are effectively non-deterministic.
- majormajor 1y agoEven after temperature=0 I believe there is some non-determinism at the chip level, similar to https://stackoverflow.com/questions/50744565/how-to-handle-non-determinism-when-training-on-a-gpu https://stackoverflow.com/questions/50744565/how-to-handle-n...
- noosphr 1y agoFrom the good old days: https://152334h.github.io/blog/non-determinism-in-gpt-4/ https://152334h.github.io/blog/non-determinism-in-gpt-4/ (that's been a short two years). Unless something has fundamentally changed since then (which I've not heard about) all sparse models are only deterministic at the batch level, rather than the sample level.