You are correct that we have two modalities projecting into a shared latent space. However, that doesn't convey anything about how to jointly train the embeddin

You are correct that we have two modalities projecting into a shared latent space. However, that doesn't convey anything about how to jointly train the embedding models, does it? That's what the two tower embedding model shows - how to jointly train the models using distance functions like cosine distance. On an aside, others, as esteemed as Yann Lecun, are talking about learning in a shared latent space between 2 modalities, but use different names - joint embeddings, because it's a different method. https://arxiv.org/abs/2307.12698 https://arxiv.org/abs/2307.12698