3 ms·

Here's a nice video by Yannick Kilcher explaning the Nystromformer: https://www.youtube.com/watch?v=m-zrcmRd7E4 https://www.youtube.com/watch?v=m-zrcmRd7E4 The

Here's a nice video by Yannick Kilcher explaning the Nystromformer: https://www.youtube.com/watch?v=m-zrcmRd7E4 https://www.youtube.com/watch?v=m-zrcmRd7E4 The benefits over regular transformers is that it is more efficient (does less operations), as the original transformer has a quadratic complexity in the number of input tokens.

tveita 6y ago
It also links to a comparison that is not in the paper, against Performer, Linformer and Reformer: https://twitter.com/tanmingxing/status/1359301186734620675 https://twitter.com/tanmingxing/status/1359301186734620675