3 ms·
llama-server also supports saving the kv cache to SSD. I had no issues with cache invalidation using pi.
by quantumleaper 2mo ago
llama-server also supports saving the kv cache to SSD. I had no issues with cache invalidation using pi.
- gcr 2mo agoTIL! When was this functionality added? It wasn’t in llamacpp when I looked in June