3 ms·
This is looking pretty good for me so far. Performance is decent with ollama on a CPU and it is surprisingly capable for a 7B model.
by _ea1k 3y ago
This is looking pretty good for me so far. Performance is decent with ollama on a CPU and it is surprisingly capable for a 7B model.
- brucethemoose2 3y agoEven a tiny Intel/AMD GPU makes a massive difference, as it makes prompt processing nearly instant vs being barely faster than generation.