4 ms·

• The GPU in the base M2 has 10 GPU cores and 3.6 tflops • The GPU in the M2 Ultra has 76 GPU cores and corresponds to 2x M2 Max, which has 13.6 tflops; so 27.

• The GPU in the base M2 has 10 GPU cores and 3.6 tflops • The GPU in the M2 Ultra has 76 GPU cores and corresponds to 2x M2 Max, which has 13.6 tflops; so 27.2 tflops [1,2] • RTX 4090 has 82.58 tflops [3] (overclocked can reach 100 tflops [4]) While more powerful NVidia cards are not "multiple orders of magnitude more". Rather it seems a 4090 is around 3-4 faster than an M2 Ultra GPU. Keep in mind that the Apple Silicon chips also have low-precission Neural Engine circuits for inference of neural nets. For the M2 Ultra they claim 31.6 tops [1]. [1] https://www.apple.com/newsroom/2023/06/apple-introduces-m2-ultra/ https://www.apple.com/newsroom/2023/06/apple-introduces-m2-u... [2] https://www.notebookcheck.net/Apple-unveils-M2-Pro-and-M2-Max-SoCs-for-Mac-mini-and-MacBook-Pro-claims-M2-Max-GPU-prowess-is-unmatched-even-by-Quadro-RTX-6000-and-RTX-3080-Ti-powered-laptops.682425.0.html https://www.notebookcheck.net/Apple-unveils-M2-Pro-and-M2-Ma... [3] https://www.techpowerup.com/gpu-specs/geforce-rtx-4090.c3889 https://www.techpowerup.com/gpu-specs/geforce-rtx-4090.c3889 [4] https://videocardz.com/newz/overclocked-nvidia-rtx-4090-gpu-hits-100-tflops-performance-with-3-15-ghz-clock https://videocardz.com/newz/overclocked-nvidia-rtx-4090-gpu-...

mosshammer 3y ago
The RTX 4090 has 330.3 TOPS with INT8 precision, so for inference workloads it is still a magnitude faster
lostmsu 3y ago
Since we are talking about AI, you should use Tensor Core Tops instead of regular flops. That number for 4090 is 330. So more than 10x of M2 Ultra
fyzix 3y ago
Max to ultra doesn't scale linearly. Max tech on YouTube covered this extensively
- joakleaf 3y ago
  Probably true that performance doesn't scale linearly (and I do remember the Max Tech analysis of M1), but I don't have the tflops number for the M2 ultra. For the M1 the stated values are: M1 Ultra [1]: 21 TFLOPs M1 Max [2]: 10.6 TFLOPS Actual performance obviously depends on the type of work-load etc. E.g. Geekbench Ultra is only 50% faster [3]: M1 Ultra Geekbench 6: 150260 M1 Max Geekbench 6: 108198 [1] https://videocardz.com/newz/despite-apples-claims-m1-ultra-gpu-is-not-as-powerful-as-rtx-3090 https://videocardz.com/newz/despite-apples-claims-m1-ultra-g... [2] https://wccftech.com/m1-max-teraflops-performance-higher-than-ps5/ https://wccftech.com/m1-max-teraflops-performance-higher-tha... [3] https://browser.geekbench.com/metal-benchmarks https://browser.geekbench.com/metal-benchmarks

mosshammer 3y ago

The RTX 4090 has 330.3 TOPS with INT8 precision, so for inference workloads it is still a magnitude faster

lostmsu 3y ago

Since we are talking about AI, you should use Tensor Core Tops instead of regular flops. That number for 4090 is 330. So more than 10x of M2 Ultra

fyzix 3y ago

Max to ultra doesn't scale linearly. Max tech on YouTube covered this extensively

joakleaf 3y ago
Probably true that performance doesn't scale linearly (and I do remember the Max Tech analysis of M1), but I don't have the tflops number for the M2 ultra. For the M1 the stated values are: M1 Ultra [1]: 21 TFLOPs M1 Max [2]: 10.6 TFLOPS Actual performance obviously depends on the type of work-load etc. E.g. Geekbench Ultra is only 50% faster [3]: M1 Ultra Geekbench 6: 150260 M1 Max Geekbench 6: 108198 [1] https://videocardz.com/newz/despite-apples-claims-m1-ultra-gpu-is-not-as-powerful-as-rtx-3090 https://videocardz.com/newz/despite-apples-claims-m1-ultra-g... [2] https://wccftech.com/m1-max-teraflops-performance-higher-than-ps5/ https://wccftech.com/m1-max-teraflops-performance-higher-tha... [3] https://browser.geekbench.com/metal-benchmarks https://browser.geekbench.com/metal-benchmarks