3 ms·
What’s iSWA? Can’t find any reference online
by vlovich123 1y ago
What’s iSWA? Can’t find any reference online
- nolist_policy 1y agointerleaved sliding window attention
- imtringued 1y agoGemma 3 has some layers with a context size of 1024 tokens and others having full length. You need to read the Gemma technical report.