3 ms·
For an in-depth review you can see: Yannic Kilcher - Big Bird: Transformers for Longer Sequences (Paper Explained) https://www.youtube.com/watch?v=WVPE62Gk3EM
by visarga 6y ago
For an in-depth review you can see:
Yannic Kilcher - Big Bird: Transformers for Longer Sequences (Paper Explained)
https://www.youtube.com/watch?v=WVPE62Gk3EM https://www.youtube.com/watch?v=WVPE62Gk3EM
- mdrabla 6y agoThanks for the link - pretty good review. Totally agree with what he says about calling it SOTA. It's definitely a very interesting approach but I wouldn't put that much weight on whether it's achieved SOTA, because of the amount of compute they threw into it.