3 ms·
I’m not sure what you mean. The words “transformer architecture” are standard across the literature.
by l33t233372 4y ago
I’m not sure what you mean. The words “transformer architecture” are standard across the literature.
- nl 4y agoThat's like saying "LSTM architecture". It says it uses a transformer but no description of how it is used. For example, the GPT architecture (which GPT 2 & 3 are slight modifications of) comprises of an embedding layer followed by 12x(self-attention/layer norm/feed forward/layer norm. That's what that GPT "transformer architecture" is, not just the transformer block itself. Strictly the architecture really also includes things like the embedding size and number of heads (which are in the GPT paper).