3 ms·
How it compares to the many hybrid SSM/Transformer architectures that are coming out now, with 2B+ weights released? I skimmed the paper, but it seems it's inno
by Manabu-eo 2y ago
How it compares to the many hybrid SSM/Transformer architectures that are coming out now, with 2B+ weights released? I skimmed the paper, but it seems it's innovation is basically combining both in the same layer, instead of alternating layers, but I see no reference to the latter.