3 ms·
Isn't the "KV Compression Strategies (FAIR)" chart showing that the fancy complex algorithm only barely beats simple topk? The commentary says that topk "degra
by jbellis 6mo ago
Isn't the "KV Compression Strategies (FAIR)" chart showing that the fancy complex algorithm only barely beats simple topk?
The commentary says that topk "degrades rapidly at low ratios" but the same can be seen for HAE (Entropy + OLS).
- jchandra 6mo agoFair point, the gap isn’t huge in that plot, and both degrade at low ratios. The difference is more in how they degrade: TopK can have sharper, localized failures, while HAE tends to be a bit more smooth. That doesn’t always show up strongly in average MSE. That said, the gains are modest right now, this is still a research prototype exploring the tradeoff, and there’s clearly more work to be done.
- bee_rider 6mo agoIs it really that fancy and complex, though? The “entropy recycling bin” seems fancy to me, but the other stuff is least squares and an SVD, these are solid workhorse numerical routines.