The M5 is a huge upgrade over the M4 for local inference. They advertise 400% and there is reason to believe this isn’t a totally BS number. They redo the GPU c

The M5 is a huge upgrade over the M4 for local inference. They advertise 400% and there is reason to believe this isn’t a totally BS number. They redo the GPU cores to avoid having to emulate certain operations at the core inner loop of LLM inference. I have an M4 and it is plenty fast enough. But honestly the local models are just not anywhere near the hosted models in quality, due to the lower parameter count, so I haven’t had much success yet.