4 ms·
It would double the size of the KV cache, which can be significant (multi-GB) at larger context sizes.
by Kubuxu 2y ago
It would double the size of the KV cache, which can be significant (multi-GB) at larger context sizes.
loading thread…
4 ms·