3 ms·
How large is the KV cache?
by lostmsu 7mo ago
How large is the KV cache?
- xbar 7mo ago0.1 GB per full-attention layer and "The model has 60 transformer layers: 45 GatedDeltaNet (linear attention) + 15 standard full attention." So, 1.5 GB.
loading thread…