3 ms·
1 to 2 tokens per second on CPU, depending on the CPU/RAM, thanks to GGML. See discussions: https://github.com/ggerganov/ggml/pull/231 https://github.com/ggerg
by jxy 3y ago
1 to 2 tokens per second on CPU, depending on the CPU/RAM, thanks to GGML.
See discussions: https://github.com/ggerganov/ggml/pull/231 https://github.com/ggerganov/ggml/pull/231
And the code: https://github.com/jploski/ggml/tree/falcon40b https://github.com/jploski/ggml/tree/falcon40b