4 ms·
New KV cache compaction technique cuts LLM memory 50x without accuracy loss
- androiddrew 8mo agoI hope this is real.
- westurner 8mo agoScholarlyArticle: "Fast KV Compaction via Attention Matching" (2025) https://arxiv.org/abs/2602.16284 https://arxiv.org/abs/2602.16284