3 ms·
I think they are orthogonal. Flash attention is just another way to compute exact attention. This work mainly concerns how to resolve memory fragmentation acr
by karmasimida 3y ago
I think they are orthogonal.
Flash attention is just another way to compute exact attention.
This work mainly concerns how to resolve memory fragmentation across different sequences
You still need to compute attention as is once you retrieve the needed key values
- wskwon 3y agoThanks for the explanation! I believe the two ideas are basically orthogonal. FlashAttention reduces memory read/writes, while PagedAttention reduces memory waste.