A lot of embedding models are built on top of T5's encoder, this offers a new option The modularity of the enc-dec approach is useful - you can insert addition

A lot of embedding models are built on top of T5's encoder, this offers a new option The modularity of the enc-dec approach is useful - you can insert additional models in between (e.g. A diffusion model), you can use different encoders for different modalities, etc