2 ms·
I'm on Windows 10, 3.5GHz AMD CPU (dual core, iGPU), 8gb RAM and I get about 3 tokens per second with the smallest model (ggml-alpaca-7b-q4).
by sourcecodeplz 3y ago
I'm on Windows 10, 3.5GHz AMD CPU (dual core, iGPU), 8gb RAM and I get about 3 tokens per second with the smallest model (ggml-alpaca-7b-q4).