4 ms·
I am running it on a 32G memory mac mini with an M2 Pro using Ollama. It runs fine, faster than I expected. The way it explains plans for solving problems, then
by mark_l_watson 2y ago
I am running it on a 32G memory mac mini with an M2 Pro using Ollama. It runs fine, faster than I expected. The way it explains plans for solving problems, then proceeding step by step is impressive.
- j0hnyl 2y agoHow many tokens per second?
- mark_l_watson 2y agoI am away from my computer, but I think it was about 10/second - not too bad.
- andrewmunsell 2y agoAnother data point: 17.6 tokens/s on an M4 Max 40 core GPU
- behnamoh 2y ago8.4 tps on M1 Pro chip with 32GM RAM (Q4 model, 18GB).