4 ms·
Jagged Flash Attention Optimization
- platers 2y agoFlash attention natively supports packing multiple variable length sequences into a single call, what is the advantage of jagged flash attention?
- deleted 2y ago[deleted]
- bbstats 2y agoIf only there was a link to a page somewhere that could answer this question for you.
- CapsAdmin 2y agoSee also https://github.com/thu-ml/SageAttention https://github.com/thu-ml/SageAttention and https://github.com/thu-ml/SpargeAttn https://github.com/thu-ml/SpargeAttn