4 ms·
> Current performant (high accuracy) LLMs have a quadratic cost (space and time) in sequence length, FlashAttention only takes polylogarithmic space overhead b
by namibj 3y ago
> Current performant (high accuracy) LLMs have a quadratic cost (space and time) in sequence length,
FlashAttention only takes polylogarithmic space overhead beyond the linear needed for attention.
Yes, quadratic compute, but that's much more tame.