20 ms·
Anyone with a comparison with Intel's deep learning boost or VNNI, which is available on avx-512 processors such as https://ark.intel.com/content/www/us/en/ark/
by kristianp 4y ago
Anyone with a comparison with Intel's deep learning boost or VNNI, which is available on avx-512 processors such as https://ark.intel.com/content/www/us/en/ark/products/213805/intel-core-i511400h-processor-12m-cache-up-to-4-50-ghz.html https://ark.intel.com/content/www/us/en/ark/products/213805/...
- brrrrrm 4y agoI don't see throughput information on Intel's AMX, but their VNNI has information: https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#avx512techs=AVX512_VNNI&ig_expand=2856 https://www.intel.com/content/www/us/en/docs/intrinsics-guid... 0.5 cycles per instruction max, 3.7GHz clock, that's 7.4e9 instructions per second. If I'm reading it right, that instruction does 16 4-wide dot products, which is ~128 ops. So ~950Gops peak in int8 precision on a server class Xeon assuming no clock throttling. (edit: flops -> ops)
- stephencanon 4y agoInt8 operations are not "flops" =)