Attention is all you need is the title of the 2017 paper that introduced the Transformer architecture. It reflects the evolution of NLP models: (RNN) —> (RNN +

Attention is all you need is the title of the 2017 paper that introduced the Transformer architecture. It reflects the evolution of NLP models: (RNN) —> (RNN + attention) —> (attention) and the idea that you don’t need the recurrent component. Just look at all elements of the sequence at once, applying varying weights (attention).