3 ms·

> random issues like the bias term I'd like to know more about this!

• Scene_Cast2 2y ago
  Basically this style of code - https://github.com/pytorch-labs/attention-gym/pull/84/files https://github.com/pytorch-labs/attention-gym/pull/84/files - has issues like this - https://github.com/pytorch/pytorch/pull/137452 https://github.com/pytorch/pytorch/pull/137452 https://github.com/pytorch/pytorch/issues/144511 https://github.com/pytorch/pytorch/issues/144511 https://github.com/pytorch/pytorch/issues/145869 https://github.com/pytorch/pytorch/issues/145869 For some higher level context, see https://pytorch.org/blog/flexattention/ https://pytorch.org/blog/flexattention/