4 ms·
Yes, in theory, of course. In practice - on a multitasking OS with input from multiple human users - it's hard to get it deterministic because of that GPU sche
by qarl 27d ago
Yes, in theory, of course.
In practice - on a multitasking OS with input from multiple human users - it's hard to get it deterministic because of that GPU scheduling thing I mentioned.
- fc417fc802 26d agoGPU scheduling only affects the result due to buggy optimizations. It's the exact same mechanism as fp rounding error on the CPU or updates to globally shared PRNG state. We use lots of buggy optimizations because they don't matter in practice in most situations (see ex -ffast-math).
- qarl 26d agoI don't know the details but apparently it has something to do with multiprocess contention for the GPU and batch sizing.
- fc417fc802 26d agoRight but regardless, it's a buggy optimization. The calculations are all fully deterministic when done "properly" and fully consistently but we almost never bother with that because it slows things down and the errors don't matter in practice 99% of the time.
- qarl 26d agoAGREED. Although whoever designed that architecture might take issue with your description as "buggy".