The reason they're good is that they're not recurrent: you can do attention over all of the sequence at once. Recurrent models had two flaws: (1) vanishing gra

The reason they're good is that they're not recurrent: you can do attention over all of the sequence at once. Recurrent models had two flaws: (1) vanishing gradient problem (2) not parallelizable