2 ms·
These stats are true, but the CPU's biggest advantage is L1, L2, and L3 cache. In particular, the 3990x, 64-core Threadripper will have 256MBs of aggregate L3
by dragontamer 7y ago
These stats are true, but the CPU's biggest advantage is L1, L2, and L3 cache.
In particular, the 3990x, 64-core Threadripper will have 256MBs of aggregate L3 cache, and 512kBs L2 cache per core (32MBs of L2 cache). Highly-optimized kernels may fit large portions of data within L3 cache and rarely even touch DDR4!
Note: Each L3 cache is only 16MBs between 4-cores. It will take some tricky programming to split a model into 16MB chunks, but if it can be done, Threadripper would be crazy fast.
True, GPUs have really fat VRAM to work with, but CPUs have really fat L3 cache and L2 cache to work with. And the CPU caches are coherent too, simplifying atomic code. GPUs do have "shared memory" and L2 caches, but they're far smaller than CPU-caches.