> H 7B is always faster than optimized Transformers > (>10%, >20% and >50% end-to-end faster than FlashAttention 50% improvement on very large sequences is gre

> H 7B is always faster than optimized Transformers > (>10%, >20% and >50% end-to-end faster than FlashAttention 50% improvement on very large sequences is great. But it is not yet transformative (pun intended..) Presumably FlashAttention already does lots of the less-than-quadratic improvement