2 ms·
Yes, but the relevance of that is escaping me. Would you mind elaborating?
by hansvm 3y ago
Yes, but the relevance of that is escaping me. Would you mind elaborating?
- namibj 3y ago> Current performant (high accuracy) LLMs have a quadratic cost (space and time) in sequence length, FlashAttention only takes polylogarithmic space overhead beyond the linear needed for attention. Yes, quadratic compute, but that's much more tame.