Luckily the use of Transformer models makes what's going on under the hood a bit more interpretable, but I think the fundamental part at which ideas are merged

Luckily the use of Transformer models makes what's going on under the hood a bit more interpretable, but I think the fundamental part at which ideas are merged is translating from CLIP text embeddings to CLIP image embeddings. The training principle of CLIP is very simple, but intuitively understanding how the diffusion prior maps between semantically similar textual and visual representations is a bit more unclear (if that's even a well-formulated question!)