That's the autoregressive aspect. The decoder aspect is that the last layer converts representations into output sequences (and the generation happens autoregre

That's the autoregressive aspect. The decoder aspect is that the last layer converts representations into output sequences (and the generation happens autoregressively, one at a time). Similarly at the last layer an encoder outputs a representation/embedding (while being able to attend to the entire sequence).