4 ms·
I haven't fully read the paper yet. Isn't the strength of Vision Transformers in unsupervised learning, meaning that the data doesn't need labels? And don't Res
by matrix2596 3y ago
I haven't fully read the paper yet. Isn't the strength of Vision Transformers in unsupervised learning, meaning that the data doesn't need labels? And don't ResNets require labeled data?
- janalsncm 3y agoIn the vision transformer paper they trained on cfar 10 and Imagenet which are supervised learning problems.
- Herring 3y agoIt's possible to adapt ResNets for unsupervised learning. A popular approach is to use self-supervised learning techniques, where the model is initially trained to predict some aspect of the data (e.g., predicting a missing part of an image) without explicit labels. This is similar to masked language modeling.