For a low-power SoC, the GPU performance is actually pretty impressive. We recently did some transformer benchmarks and the inference performance of the M1 Max

For a low-power SoC, the GPU performance is actually pretty impressive. We recently did some transformer benchmarks and the inference performance of the M1 Max is almost half that of an RTX3090: https://explosion.ai/blog/metal-performance-shaders https://explosion.ai/blog/metal-performance-shaders However the SoC only uses 31W when posting that performance.