"Defeating Nondeterminism in LLM Inference" ( https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference https://thinkingmachines.ai/blog/defeat

"Defeating Nondeterminism in LLM Inference" ( https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference https://thinkingmachines.ai/blog/defeating-nondeterminism-in...) has a repo: https://github.com/thinking-machines-lab/batch_invariant_ops https://github.com/thinking-machines-lab/batch_invariant_ops which seems to have eventually been merged into vllm: https://docs.vllm.ai/en/latest/features/batch_invariance/ https://docs.vllm.ai/en/latest/features/batch_invariance/ So you can get determinism locally. On a cursory search I wasn't able to find any LLM provider advertising determism; if you need it for research you might have to rent a dedicated GPU pod and run vllm there with the appropriate settings.