If you look at the classical [transformer architecture picture](https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)#/media/File:The-Transform

If you look at the classical [transformer architecture picture](https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)#/media/File:The-Transformer-model-architecture.png https://en.wikipedia.org/wiki/Transformer_(deep_learning_arc...) there is an "encoder" tower on the left and a "decoder" tower on the right. - Bert is encoder only. - GPT is decoder only. - T5 uses both the encoder and the decoder.