3 ms·
I don't believe it's the actual first but this is pretty awesome. too bad its facebook :/
by silence48 5y ago
I don't believe it's the actual first but this is pretty awesome. too bad its facebook :/
- leirbagarc 5y agoI think the big achievement is how it surpassed in performance previous models for each individual modality.
- gillesjacobs 5y agoThat's not true for NLU at least. It is on par with 2018's RoBERTa on GLUE, many larger and advanced language models came after. It is still great work though, a robust masking representation architecture that works across modalities.
- kuu 5y agoThey mention that on the article: > We apply data2vec separately to speech, images and text and it outperformed the previous best single-purpose algorithms for computer vision and speech and it is competitive on NLP tasks.