3 ms·
And still today we spend a great deal of effort trying to make our randomly-sampled LLM outputs reproducibly deterministic: https://thinkingmachines.ai/blog/de
by Imnimo 11mo ago
And still today we spend a great deal of effort trying to make our randomly-sampled LLM outputs reproducibly deterministic:
https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/ https://thinkingmachines.ai/blog/defeating-nondeterminism-in...
- heavenlyblue 11mo agocan't you just save the seed?
- bfkwlfkjf 11mo agoMy understanding is that because GPUs do operations in a highly parallelized fashion, and because float point operations aren't commutative, then once you're using GPUs the seed isn't enough, no. You'd need the seed plus the specific order in which each of intermediate steps of the calculation was finished by the various streaming multiprocessors.