I think your understanding of this method is wrong. This work is about unifying the training objective across modalities, not training on multiple modalities si

I think your understanding of this method is wrong. This work is about unifying the training objective across modalities, not training on multiple modalities simultaneously. A single data2vec model is not meant to take different types of data as input, at least as far as I understood it. Directly from the paper: "Our work does not perform multimodal training but aims to unify the learning objective for self-supervised learning in different modalities"