2 ms·
I use both. Apple’s higher end hardware works well for local LLM inference thanks to its unusually high memory bandwidth and frugal energy consumption. But for
by ein0p 3y ago
I use both. Apple’s higher end hardware works well for local LLM inference thanks to its unusually high memory bandwidth and frugal energy consumption. But for serious work you need CUDA, no ifs or buts.