6 ms·
have you actually benchmarked that? I think (someone please correct me if I'm way off here) the AMX instructions can hit ~2.8tflops (fp16) per co-processor and
by brrrrrm 4y ago
have you actually benchmarked that? I think (someone please correct me if I'm way off here) the AMX instructions can hit ~2.8tflops (fp16) per co-processor and there are 2 on the 7-core M1. That's 5.6tflops vs the 4.6tflops the GPU can hit.
- johndough 4y agoOften the limiting factor is memory bandwidth instead of raw FLOPS, so dealing with 4 times larger data types (FP64 vs FP16) is a disadvantage.
- brrrrrm 4y agoto clarify: I am comparing FP16 performance, which both the GPU and AMX have native support for. FP64 is also supported by AMX, making it quite an impressive region of silicon.
- my123 4y agoYeah that's within the M1 family, but get within dGPUs and it doesn't even come close. 30Tflops for a 3080 for vector FP32, but 119Tflops FP16 dense with FP16 accumulate, 59.5 with FP32 accumulate, and if you exploit sparsity then that can go even higher.
- brrrrrm 4y agoAh yes, I misunderstood your original comment