3 ms·
Knowing the performance is interesting. Apparently it's 1-3 tokens/second.
by skybrian 9mo ago
Knowing the performance is interesting. Apparently it's 1-3 tokens/second.
- kgeist 9mo agoikllama.cpp is a fork of llama.cpp which specializes on CPU inference, some benchmarks from 1 year ago: https://github.com/ikawrakow/ik_llama.cpp/discussions/164 https://github.com/ikawrakow/ik_llama.cpp/discussions/164