3 ms·
You can run Ollama on your aging laptop today, but at how many tokens per second and at how many parameters? Most consumer PC hardware isn't going to have a ver
by runjake 3y ago
You can run Ollama on your aging laptop today, but at how many tokens per second and at how many parameters? Most consumer PC hardware isn't going to have a very capable GPU. But, I certainly anticipate "AI accelerators" in consumer PCs, in the short term, ala AI PC NPUs.
FWIW, I basically leave Ollama running on my M3 Max and 24/7 and use it regularly, and it's ridiculously fast, but: M3 Max with lots of RAM and a crazy memory busy.