3 ms·

* If you remove the softmax from the attention formula, you end up with linear attention* Sorry, what?

• tripplyons 2y ago
  Here is a paper explaining it: https://arxiv.org/abs/2006.16236 https://arxiv.org/abs/2006.16236