3 ms·
When one says "attention is all you need" the implication is that some believe that you need something more than just attention. What is that something which h
by nafey 3y ago
When one says "attention is all you need" the implication is that some believe that you need something more than just attention. What is that something which has been demonstrated as unneeded? Is it a theory of how language works?
- kylewatson 3y agoI think this will help: https://medium.datadriveninvestor.com/attention-in-rnns-321fbcd64f05 https://medium.datadriveninvestor.com/attention-in-rnns-321f...
- trenchgun 3y agoRecursion. Before transformers attention was used in recurrent neural networks. "attention is all you need" showed that you can just drop the recursion and just use attention, and the outcome is that you get a very nicely parallelizable architechture, allowing more efficient training.
- Sammi 3y agoFinally someone cut to chase. Thanks you!