3 ms·
No, this is a new iteration on the AIAYN idea.
by bigdict 6y ago
No, this is a new iteration on the AIAYN idea.
- elcomet 6y agoWhat's this ?
- ot 6y ago"Attention is all you need"
- bigdict 6y agoAttention is all you need is the title of the 2017 paper that introduced the Transformer architecture. It reflects the evolution of NLP models: (RNN) —> (RNN + attention) —> (attention) and the idea that you don’t need the recurrent component. Just look at all elements of the sequence at once, applying varying weights (attention).
- deleted 6y ago[deleted]
- the8472 6y agoAlso an instance of The Bitter Lesson