3 ms·
Unless you have something exotic, the randomness that's adding to a computer is a combination of how it's configured combined with a pseudo-random number genera
by joe_the_user 10d ago
Unless you have something exotic, the randomness that's adding to a computer is a combination of how it's configured combined with a pseudo-random number generator. I assume the system adds entropy to the generator regularly but all you need to do is fix the various supposedly random inputs and you can get full determinism even without zero temperature.
- qarl 10d agoYes, in theory, of course. In practice - on a multitasking OS with input from multiple human users - it's hard to get it deterministic because of that GPU scheduling thing I mentioned.
- fc417fc802 10d agoGPU scheduling only affects the result due to buggy optimizations. It's the exact same mechanism as fp rounding error on the CPU or updates to globally shared PRNG state. We use lots of buggy optimizations because they don't matter in practice in most situations (see ex -ffast-math).
- qarl 10d agoI don't know the details but apparently it has something to do with multiprocess contention for the GPU and batch sizing.
- fc417fc802 9d agoRight but regardless, it's a buggy optimization. The calculations are all fully deterministic when done "properly" and fully consistently but we almost never bother with that because it slows things down and the errors don't matter in practice 99% of the time.
- qarl 9d agoAGREED. Although whoever designed that architecture might take issue with your description as "buggy".