2 ms·There is a bug in llama-cpp for qwen/gemma models, use vLLM insteadby verdverm 5mo agoThere is a bug in llama-cpp for qwen/gemma models, use vLLM insteadpdyc 5mo agowhat bug and it affects what?verdverm 5mo agoit's a prompt cache invalidation bug that causes all input to be reprocessed instead of getting preloaded There are other reasons to prefer vllm to llama-cpp as well