4 ms·
how can the multicore AVX implementation do a dot product (for arrays much larger than cache) at 340 GB/s on a system with RAM bandwidth < 50 GB/s
by rnrn 2y ago
how can the multicore AVX implementation do a dot product (for arrays much larger than cache) at 340 GB/s on a system with RAM bandwidth < 50 GB/s
- alecco 2y agoI think the post is a bit disingenuous. But about bandwidth, matrix multiplications happen mostly in cache and that has a lot more bandwidth than RAM. Blocks of the matrix are loaded to cache (explicitly in CUDA) and used multiple times there. I'd exploit the better multi-level cache hierarchy in CPUs and make the code NUMA aware. But still I wouldn't bet against a recent GPU card.
- rnrn 2y ago> But about bandwidth, matrix multiplications happen mostly in cache and that has a lot more bandwidth than RAM. Blocks of the matrix are loaded to cache (explicitly in CUDA) and used multiple times there. The post is about dot product, not matrix multiply. Dot product has no data reuse
- rnrn 2y agoAnswer: it can’t. The author has updated the post with corrected AVX measurements, with the original ~340 GB/s revised down to 31.7 GB/s. (Thanks CowFreedom)