4 ms·
AMX is indeed very nice for FP64 where customer GPUs aren't an alternative at all. However, for lower precisions (which is what deep learning uses), you're muc
by my123 4y ago
AMX is indeed very nice for FP64 where customer GPUs aren't an alternative at all.
However, for lower precisions (which is what deep learning uses), you're much better off with a GPU.
- brrrrrm 4y agohave you actually benchmarked that? I think (someone please correct me if I'm way off here) the AMX instructions can hit ~2.8tflops (fp16) per co-processor and there are 2 on the 7-core M1. That's 5.6tflops vs the 4.6tflops the GPU can hit.
- johndough 4y agoOften the limiting factor is memory bandwidth instead of raw FLOPS, so dealing with 4 times larger data types (FP64 vs FP16) is a disadvantage.
- brrrrrm 4y agoto clarify: I am comparing FP16 performance, which both the GPU and AMX have native support for. FP64 is also supported by AMX, making it quite an impressive region of silicon.
- my123 4y agoYeah that's within the M1 family, but get within dGPUs and it doesn't even come close. 30Tflops for a 3080 for vector FP32, but 119Tflops FP16 dense with FP16 accumulate, 59.5 with FP32 accumulate, and if you exploit sparsity then that can go even higher.
- brrrrrm 4y agoAh yes, I misunderstood your original comment