3 ms·
Speculative decoding[1]? [1]: https://github.com/ggml-org/llama.cpp/blob/master/docs/speculative.md#draft-model-draft https://github.com/ggml-org/llama.cpp/blo
by genpfault 7mo ago
Speculative decoding[1]?
[1]: https://github.com/ggml-org/llama.cpp/blob/master/docs/speculative.md#draft-model-draft https://github.com/ggml-org/llama.cpp/blob/master/docs/specu...