4 ms·
Just use llama.cpp or Unsloth Studio which wraps it, I don't know why anyone use Ollama anymore.
by satvikpendem 5mo ago
Just use llama.cpp or Unsloth Studio which wraps it, I don't know why anyone use Ollama anymore.
- verdverm 5mo agoI switched from llama.cpp to vLLM because of prompt cache bugs in qwen/gemma models This is a good starting issue with a bunch of linked/related https://github.com/ggml-org/llama.cpp/issues/22746 https://github.com/ggml-org/llama.cpp/issues/22746