3 ms·
> They are in the unique position to enable local AI tools Nothing special about Apple with regards to AI. M2 beats x86 in power efficiency, but not significan
by xnx 4y ago
> They are in the unique position to enable local AI tools
Nothing special about Apple with regards to AI. M2 beats x86 in power efficiency, but not significantly better than other ARM processors.
- shepherdjerred 4y agoI think the parent is referring to the Apple Neural Engine in Apple Silicon, which aren't widely used today (as far as I know)
- gcr 4y agoSome workloads on M1 absolutely smash other ARM processors in part because of M1's special-purpose hardware. In particular, the undocumented AMX chip is really nice for distance matrix calculations, vector search, embeddings, etc. Non-scientific example: for inference, whisper.cpp links with Accelerate.framework to do fast matrix multiplies. On M1, one configuration gets ~6x realtime speed, but on a very beefy AWS Gravatron processor, the same configuration only achieves 0.5x realtime, even after choosing an optimal threadcount, even linking with NEON-optimized BLAS. (Maybe I'm doing something wrong though).