4 ms·
And catastrophically slower. I think a lot of people who are downloading llama.cpp to run on their M-series MacBook Pro or spending $$$ on high-RAM Mac Studios
by modeless 3y ago
And catastrophically slower. I think a lot of people who are downloading llama.cpp to run on their M-series MacBook Pro or spending $$$ on high-RAM Mac Studios for AI are not actually aware that Nvidia GPUs are often literally ten times faster for the same applications. You'll almost always be better off either running on a local gaming GPU for things that fit or renting GPUs in the cloud for things that don't, just because the performance gap is so massive.