3 ms·
That's like saying "LSTM architecture". It says it uses a transformer but no description of how it is used. For example, the GPT architecture (which GPT 2 & 3
by nl 4y ago
That's like saying "LSTM architecture". It says it uses a transformer but no description of how it is used.
For example, the GPT architecture (which GPT 2 & 3 are slight modifications of) comprises of an embedding layer followed by 12x(self-attention/layer norm/feed forward/layer norm. That's what that GPT "transformer architecture" is, not just the transformer block itself.
Strictly the architecture really also includes things like the embedding size and number of heads (which are in the GPT paper).