4 ms·
Benchmarks for what you can do on CPU alone. https://github.com/ggerganov/llama.cpp/issues/34 https://github.com/ggerganov/llama.cpp/issues/34 An M1 Max does
by kiratp 3y ago
Benchmarks for what you can do on CPU alone.
https://github.com/ggerganov/llama.cpp/issues/34 https://github.com/ggerganov/llama.cpp/issues/34
An M1 Max does 100ms per token. A 64 core threadripper about 33ms per token.