2 ms·
I suspect that this has a high chance of running afoul of Ahmdal’s Law. Even if you can parallelise the bulk of the computation, the serial parts remain single-
by jiggawatts 3y ago
I suspect that this has a high chance of running afoul of Ahmdal’s Law. Even if you can parallelise the bulk of the computation, the serial parts remain single-threaded and start to dominate the total runtime.
- taneq 3y agoI don’t think the serial parts of ML training are Python’s fault, are they? It’s all “operation B depends on the output of operation A”.