3 ms·
On the other hand ollama supports iSWA for Gemma 3 while llama.cpp doesn't. iSWA reduces kv cache size to 1/6.
by nolist_policy 1y ago
On the other hand ollama supports iSWA for Gemma 3 while llama.cpp doesn't. iSWA reduces kv cache size to 1/6.
- vlovich123 1y agoWhat’s iSWA? Can’t find any reference online
- nolist_policy 1y agointerleaved sliding window attention
- imtringued 1y agoGemma 3 has some layers with a context size of 1024 tokens and others having full length. You need to read the Gemma technical report.