3 ms·
I thought that was pioneered in the paper that came out of Google?
by user_named 3y ago
I thought that was pioneered in the paper that came out of Google?
- riku_iki 3y agoAttention was first proposed in this paper: https://arxiv.org/pdf/1409.0473.pdf https://arxiv.org/pdf/1409.0473.pdf and it is referenced from transformer paper.
- VeejayRampay 3y agoat this rate no one ever invents anything cause everything is built on math that some of the elements came out of somewhere else doesn't change the fact that Google came up with an impressive and game-changing piece of tech
- riku_iki 3y agoIt was not impressive at that time, because didn't achieve any quality breakthroughs. Later adapters of transformers like GPT and Bert achieved it with impressive results.