4 ms·
It's very exciting in the local LLMs space, where the unified memory allows fast inference of large models.
by stevenhuang 3y ago
It's very exciting in the local LLMs space, where the unified memory allows fast inference of large models.
- dagaci 3y agoI read that the bandwidth is 50% lower on the M3 combined with a lower CPU core coun. This reduction will impact inferrence performance. It maybe better to stick with M2 series if you really really think spending $10k on a laptop mac to do inferrence slowly (but faster than a plain PC of course) make sense.