3 ms·
> all the non-determinism comes from the seed not fully true, when using floating point the order of operations matters, and it can vary slightly due to parall
by dist-epoch 2y ago
> all the non-determinism comes from the seed
not fully true, when using floating point the order of operations matters, and it can vary slightly due to parallelism. I've seen LLMs return different outputs with the same seed.
- onionisafruit 2y agoThat’s an interesting observation. Usually we try to control that, but with LLMs the non-determinism is fine. It seems like that would make it hard to unit test LLM code, but they seem to be managing.
- kerkeslager 2y agoOh, that's really interesting. Hadn't thought of that.