5 ms·
The 3090 also can do fp16 and the M1 series only supports fp32, so the M1 series of chips basically needs more RAM for the same batch sizes. So it isn't an Ora
by apohn 5y ago
The 3090 also can do fp16 and the M1 series only supports fp32, so the M1 series of chips basically needs more RAM for the same batch sizes. So it isn't an Oranges to Oranges comparison.
Back when that M1 MAX vs 3090 blog post was released, I ran those same tests on the M1 Pro (16GB), Google Colab Pro, and free GPUs (RTX4000, RTX5000) on the Paperspace Pro plan.
To make a long story short, I don't think buying any M1 chip make senses if your primary purpose is Deep Learning. If you are just learning or playing around with DL, Colab Pro and the M1 Max provide similar performance. But Colab Pro is ~$10/month, and upgrading any laptop to M1-Max is at least $600.
The "free" RTX5000 on Paperspace Pro (~$8 month) is much faster (especially with fp16 and XLA) than M1 Max and Colab Pro, albeit the RTX5000 isn't always available. The free RTX4000 is also a faster than M1 Max, albeit you need to use smaller batch sizes due to 8GB of VRAM.
If you assume that M1-Ultra doubles the performance of M1-Max in similar fashion to how the M1-Max seems to double the gpu performance of the M1-Pro, it still doesn't make sense from a cost perspective. If you are a serious DL practitioner, putting that money towards cloud resources or a 3090 makes a lot more sense than buying the M1-Ultra.
- mehmetoguzderin 5y ago> The 3090 also can do fp16 and the M1 series only supports fp32 Apple Silicon (including base M1) actually has great FP16 support at the hardware level, including conversions. So it is wrong to say it only supports FP32.
- oneplane 5y agoI'm not sure if he was talking about the ML engine, the ARM cores, the microcode, the library or the OS. But it does indeed have FP16 in the Arm cores.
- apohn 5y agoI should have been more clear. I didn't mean the hardware, but the speedup you get from using mixed precision in something like Tensorflow with an NVIDIA GPU.
- inkyoto 5y agoFP16 is supported in M1 GPU's and Neural Engines through the CoreML framework. From https://coremltools.readme.io/docs/typed-execution https://coremltools.readme.io/docs/typed-execution : > The Core ML runtime dynamically partitions the network graph into sections for the Apple Neural Engine (ANE), GPU, and CPU, and each unit executes its section of the network using its native type to maximize its performance and the model’s overall performance. The GPU and ANE use float 16 precision, and the CPU uses float 32. Also, this exploration (https://tlkh.dev/benchmarking-the-apple-m1-max#heading-neural-engine https://tlkh.dev/benchmarking-the-apple-m1-max#heading-neura...) reports the 5.1-5.3 TFLOPS FP16 ballpark performance.
- apohn 5y agoThanks. At least when I ran the benchmarks with Tensorflow, using mixed precision resulted in the CPU being used for training instead of the GPU on the M1 Pro. So if the hardware is there for fp16 and they will implement the software support for DL frameworks, that will be great.
- mehmetoguzderin 5y agoYes, unfortunately, the software is to blame for the time being, and I also ran into issues myself. :\ Hope they catch up to what the hardware delivers well, including both the GPU and the Neural Engine.