4 ms·
So that would mean it's not "really" a 1M context window. I guess this is more plausible than a linear architecture like MAMBA or GDN.
by cubefox 8mo ago
So that would mean it's not "really" a 1M context window. I guess this is more plausible than a linear architecture like MAMBA or GDN.