The PyTorch documentation has an entire section about how to make your code deterministic. In my experience, the performance difference is negligible. https://

The PyTorch documentation has an entire section about how to make your code deterministic. In my experience, the performance difference is negligible. https://pytorch.org/docs/stable/notes/randomness.html#avoiding-nondeterministic-algorithms https://pytorch.org/docs/stable/notes/randomness.html#avoidi... Unfortunately, determinism across devices or even driver versions is not that easy. You'd have to write your own BLAS kernels using only basic operations, which are guaranteed to follow IEEE 754 semantics. https://docs.nvidia.com/cuda/floating-point/index.html https://docs.nvidia.com/cuda/floating-point/index.html One gotcha are fused multiply-adds, which the compiler may or may not introduce, so you have to wrap all your floating point operations with __fma* intrinsics to make sure the compiler does not interpret them differently.