4 ms·
FlashAttention is completely orthogonal to this. This work is about speeding up the computation of Q, K and V vectors while FlashAttention is about speeding up
by lalaland1125 2y ago
FlashAttention is completely orthogonal to this. This work is about speeding up the computation of Q, K and V vectors while FlashAttention is about speeding up the attention algorithm itself.
You could combine the two.
- toxik 2y agoThen they should have. It isn’t clear to be that this change is measurable when the code is optimized. Even in the presented numbers the difference seems meh at times.