3 ms·
Attention was first proposed in this paper: https://arxiv.org/pdf/1409.0473.pdf https://arxiv.org/pdf/1409.0473.pdf and it is referenced from transformer paper.
by riku_iki 3y ago
Attention was first proposed in this paper: https://arxiv.org/pdf/1409.0473.pdf https://arxiv.org/pdf/1409.0473.pdf and it is referenced from transformer paper.
- VeejayRampay 3y agoat this rate no one ever invents anything cause everything is built on math that some of the elements came out of somewhere else doesn't change the fact that Google came up with an impressive and game-changing piece of tech
- riku_iki 3y agoIt was not impressive at that time, because didn't achieve any quality breakthroughs. Later adapters of transformers like GPT and Bert achieved it with impressive results.