>Whereas with a LLM if we control the seed we can 100% reproduce the same result No, you can't. For the latest GPT models and the way they are run, this doesn'

>Whereas with a LLM if we control the seed we can 100% reproduce the same result No, you can't. For the latest GPT models and the way they are run, this doesn't work anymore, making the experiment completely illogical. Some of the reasons are explained here pretty well: https://152334h.github.io/blog/non-determinism-in-gpt-4/ https://152334h.github.io/blog/non-determinism-in-gpt-4/