3 ms·
I don't know the details but apparently it has something to do with multiprocess contention for the GPU and batch sizing.
by qarl 27d ago
I don't know the details but apparently it has something to do with multiprocess contention for the GPU and batch sizing.
- fc417fc802 26d agoRight but regardless, it's a buggy optimization. The calculations are all fully deterministic when done "properly" and fully consistently but we almost never bother with that because it slows things down and the errors don't matter in practice 99% of the time.
- qarl 26d agoAGREED. Although whoever designed that architecture might take issue with your description as "buggy".