4 ms·
Who is already using speculative decoding? I haven't seen anything about it in the llama.cpp or ollama docs.
by MacsHeadroom 2y ago
Who is already using speculative decoding? I haven't seen anything about it in the llama.cpp or ollama docs.
- eshoyuan 2y agohttps://github.com/ggerganov/llama.cpp/tree/master/examples/speculative https://github.com/ggerganov/llama.cpp/tree/master/examples/...