4 ms·

PyTorch 2.2: FlashAttention-v2 integration, AOTInductor

• two_in_one 3y ago
  > now supports FlashAttention-2, yielding around 2x speedups > torch.compile improvements so far 2.1 didn't work well with MoE GPT, at least in my implementation, due to dynamism in data flow. will check how 2.2 does