I guess it depends on how you're defining "great" LLM performance and "CPU performance supremacy versus Apple" Personally I would say "borderline-acceptable pe

I guess it depends on how you're defining "great" LLM performance and "CPU performance supremacy versus Apple" Personally I would say "borderline-acceptable performance" with a 7B parameter model at 4-bit quantisation is nowhere near "great" or "supremacy". You can fit that on ~7GB of RAM, using your grandpa's 2016-era nvidia 1080. To take the ARM-CPU-LLM-performance crown from Apple you have to beat a fully equipped M2 Max, which provides 96GB of memory with 400GB/s of bandwidth. That's enough to run 70B parameter models in 4-bit quantisation, and it can go toe-to-toe with a dual-nvidia-4090 system that draws a kilowatt of power.