4 ms·
Who is this :think: But no, FlashAttention already solved the memory requirements of attention. RingAttention is primarily useful for parallelizing across the
by chillee 3y ago
Who is this :think:
But no, FlashAttention already solved the memory requirements of attention. RingAttention is primarily useful for parallelizing across the sequence component.
- casercaramel144 3y agoIt's camel. How do you do matrix vector attention without keeping the full matrix in cache, surely you don't just load unload it a million times