4 ms·
What about ALiBi and Sliding Window Attention? Additionally Apple researchers seem to be playing with "Attention Free" variants.
by razodactyl 3y ago
What about ALiBi and Sliding Window Attention?
Additionally Apple researchers seem to be playing with "Attention Free" variants.