2 ms·
>LLMs, by their very nature are probabilistic. So are compilers, but people still successfully use them. Compilers and LLMs can both be made deterministic but
by charcircuit 1y ago
>LLMs, by their very nature are probabilistic.
So are compilers, but people still successfully use them. Compilers and LLMs can both be made deterministic but for performance reasons it's convenient to give up that guarantee.
- WD-42 1y agoWhat are these non deterministic compilers I keep hearing about, honestly curious.
- charcircuit 1y agoFor example looping over the files in a directory can happen in a different order depending on the order the files were created in. If you are linking a bunch of objects the order typically matters. If the compiler is implemented correctly the resulting binary should functionally be the same but the binary itself may not be exactly the same. Or even when implemented correctly you will see cases where different objects can be the one to define a duplicate symbol depending on their relative order.
- ModernMech 1y agoThat's not nondeterminism though, you've changed the input (the order of the files). Nondeterminism would be if the binary changes despite the files being in the same order. If the binary is the same holding fixed the order of the files, then the output is deterministic.
- PhunkyPhil 1y agoGCC can use randomized branch prediction.
- hn_acc1 1y agoAIUI, if you made an LLM deterministic, every mostly-similar prompt would return the same result (i.e. access the same training data set) and if that's wrong, the LLM is just plain broken for that example. Hacked-in "temperature" (randomness) is the only way to hopefully get a correct result - eventually.