4 ms·
Standard Transformer KV caches are empirically quite sparse. I wonder if they've made some fix along those lines
by JackYoustra 2y ago
Standard Transformer KV caches are empirically quite sparse. I wonder if they've made some fix along those lines