3 ms·
It's close but not fully deterministic. I remember seeing a theory that if the system is considering multiple possible completions that have the same rounded sc
by AgentME 4y ago
It's close but not fully deterministic. I remember seeing a theory that if the system is considering multiple possible completions that have the same rounded score percentage, then its choice between them is nondeterministic.
- teebs 4y agoIt's likely because GPU calculations are non-deterministic and small differences in floating point numbers could lead to different outcomes (either in the way you described or somewhere deeper in the model)
- janalsncm 4y ago> GPU calculations are non-deterministic Tensorflow is non-deterministic for some operations due to thread scheduling. PyTorch doesn’t have this issue.
- coolness 4y agoSome of the underlying CuDNN algorithms have nondeterministic implementations, which applies to PyTorch as well. See https://pytorch.org/docs/stable/notes/randomness.html https://pytorch.org/docs/stable/notes/randomness.html