3 ms·
This is a SaaS problem, not a LLM problem. If you have a local LLM that nobody is upgrading behind your back, it will calculate the same thing on the same input
by kazinator 1y ago
This is a SaaS problem, not a LLM problem. If you have a local LLM that nobody is upgrading behind your back, it will calculate the same thing on the same inputs.
Unless there is a bug somewhere, like using uninitialized memory, the flaoting-point calculations and the token embedding and all the rest do the same thing each time.
- Cilvic 1y agoSo could SaaS LLM or cloud/api LLMs not offer this as an option? A guarantee that the "same prompt" will always produce the same result. Also the way I usually interpret this "non-deterministic" a bit "broader". Say i have have slightly different prompts "what's 2+2?" vs. "can you please tell me what's 2 plus 2" or even "2+2=?" or "2+2" for most applications it would be useful if they all produce the same result
- alphan0n 1y agoThe form of the question determines the form of the outcome, even if the answer is the same. Asking the same question in a different way should result in the adherence to the form of the question. 2+2 is 4 2 plus 2 is 4 4=2+2 4 Having the LLM pass the input to a tool (python) will result in deterministic output.
- nativeit 1y agoDoesn’t that imply that LLMs are just “if then, then that” but bigger?