3 ms·

what bug and it affects what?

• verdverm 3mo ago
  it's a prompt cache invalidation bug that causes all input to be reprocessed instead of getting preloaded There are other reasons to prefer vllm to llama-cpp as well