I don't see throughput information on Intel's AMX, but their VNNI has information: https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#avx5

I don't see throughput information on Intel's AMX, but their VNNI has information: https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#avx512techs=AVX512_VNNI&ig_expand=2856 https://www.intel.com/content/www/us/en/docs/intrinsics-guid... 0.5 cycles per instruction max, 3.7GHz clock, that's 7.4e9 instructions per second. If I'm reading it right, that instruction does 16 4-wide dot products, which is ~128 ops. So ~950Gops peak in int8 precision on a server class Xeon assuming no clock throttling. (edit: flops -> ops)