3 ms·
LLMs are deterministic if you force a seed or disable sampling. They however do not guarantee that small input changes will cause small output changes.
by marcinzm 2y ago
LLMs are deterministic if you force a seed or disable sampling. They however do not guarantee that small input changes will cause small output changes.
- Jensson 2y ago> LLMs are deterministic if you force a seed or disable sampling Not with todays GPU's, you would need to run it locally with special GPU settings or run it on your CPU to ensure it is deterministic.