3 ms·
cough Apple Silicon cough cough Unified memory is honestly a really, really good idea. Supposedly, if you have professional Apple hardware experts hand-optimi
by LoganDark 3y ago
cough Apple Silicon cough cough
Unified memory is honestly a really, really good idea.
Supposedly, if you have professional Apple hardware experts hand-optimize a machine-learning model, then Apple's most powerful M3 chips can compete against, say, a typical/unoptimized PyTorch implementation of the same model on a 4090.
Except the M-series chips can currently have up to 8 times the memory of the 4090, and they don't have to worry about shuffling things in and out of VRAM. The GPU can just work with any of the data that the CPU already has.
- godelski 3y agoI'd be hard pressed to buy a comparison to a 4090. But yeah, things would be better if they didn't "dump everything into VRAM"
- LoganDark 3y agoThey're comparing incredibly hand-optimized MLX inference on the Mac to off-the-shelf stuff on the 4090. So I'd believe it just because the 4090's being given an artificial disadvantage, and the Mac is being given meticulous hand-optimization by the foremost experts on the very platform it runs on. In other words, the Mac is being given a huge optimization advantage compared to the 4090. And it's roughly competitive. Even that is no mean feat. Source: https://appleinsider.com/articles/23/12/13/apple-silicon-m3-pro-blows-away-nvidia-rtx-4090-gpu-in-ai-benchmark https://appleinsider.com/articles/23/12/13/apple-silicon-m3-...