3 ms·
> LLMs are not deterministic They can be. The cloud-hosted LLMs add a gratuitous randomization step to make the output seem more human. (In vein with the moron
by otabdeveloper4 2y ago
> LLMs are not deterministic
They can be. The cloud-hosted LLMs add a gratuitous randomization step to make the output seem more human. (In vein with the moronic idea of selling LLM's as sci-fi human-like assistants.)
But you don't have to add those randomizations. Nothing much is lost if you don't. (Output from my self-hosted LLM's is deterministic.)
- CharlesW 2y agoEven at temperature = 0, LLM output is not guaranteed to be deterministic. https://www.vincentschmalbach.com/does-temperature-0-guarantee-deterministic-llm-outputs/ https://www.vincentschmalbach.com/does-temperature-0-guarant...