3 ms·
The Attention Is All You Need paper is where Transforms were introduced: We propose a new simple network architecture, the Transformer, based solely on attenti
by nl 8y ago
The Attention Is All You Need paper is where Transforms were introduced:
We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.
- return1 8y agoYup, and the state of the art BERT and gpt-2 are both based on transformers.