3 ms·
that is why I said >If you control the randomness involved
by qsera 8mo ago
that is why I said
>If you control the randomness involved
- basch 8mo agoWhich you essentially cannot do It is inherently randomized
- qsera 8mo agoBut isn't the link shared by that comment doing exactly that https://sulbhajain.medium.com/why-llms-arent-truly-deterministic-7cc7b451fdab https://sulbhajain.medium.com/why-llms-arent-truly-determini... >The Thinking Machines research team showed it’s possible to fix this. They built batch-invariant kernels for RMSNorm, matrix multiplication, and attention, integrating them into the open-source inference engine vLLM. >The outcome: 1,000 identical prompts, 1,000 identical outputs. Perfect reproducibility. ??