4 ms·
Having run ollama on CPU: Yes, it's just slower. Not even intolerably slow IMO, though I used small models and don't mind some turnaround time.
by yjftsjthsd-h 2y ago
Having run ollama on CPU: Yes, it's just slower. Not even intolerably slow IMO, though I used small models and don't mind some turnaround time.
- jart 2y agoI've seen llamafile go about 10x faster on CPU if you try it.
- yjftsjthsd-h 2y agoReally! Same model and everything? I guess I need to go benchmark them - 10x would legit obviate GPU for me