3 ms·
Low overhead in the sense that matrix multiplication is almost the only algorithm that is able to reach computational throughput values very close to the theore
by adrian_b 2y ago
Low overhead in the sense that matrix multiplication is almost the only algorithm that is able to reach computational throughput values very close to the theoretical maximum for a given hardware.
Good CPUs and GPUs have a throughput in Flop/s for matrix multiplication that is between 60% and 90% of the maximum possible throughput, with many (especially the CPUs) reaching values towards the high end of that range.
As shown in the article, the AMD GPUs attain only slightly less than 50% (for BF16; for FP8 the AMD efficiency is even less than 40%).
Such a low efficiency for the most important operation is not acceptable.