3 ms·
CNNs are not SoTA anymore when it comes to large models, and also are not used to provide interpretations of images as text, but rather to classify, do semantic
by bigmadshoe 4mo ago
CNNs are not SoTA anymore when it comes to large models, and also are not used to provide interpretations of images as text, but rather to classify, do semantic segmentation, etc.
- tehjoker 4mo agoCan you say more about that? I haven't kept up.
- crypto420 4mo agoCNNs excel in vision tasks where you have limited compute, limited memory, limited data, and want something that works super well and quick. People usually don't hook CNNs up to a transformer to get language understanding either, you have to train bespoke CNNs for specific tasks ViTs excel where you're unbounded in compute + data and also want text understanding or have a conversation about an image
- bonoboTP 4mo agoThese are vibes. ViT has been shown to work fine on small data with proper hyperparam and most of what you mention is actually doable just fine with the other architecture as well.
- bonoboTP 4mo agoCNNs are fine when trained with a good recipe. There are very few good studies comparing them with proper hyperparam search and all the training tricks applied consistently. Transformers are good but ViT vs CNN is not some settled issue. Transformers are more hyped and more popular with the tech enthusiasts who just read forums and news, but if you need stuff done, CNNs are still great.
- bigmadshoe 4mo agoI agree, but since we're talking about imagine understanding with text output, clearly a CNN is unsuitable. My previous comment was overly reductive and CNNs can still be SoTA depending on your performance metrics. I spent the earlier part of my career training CNNs, and they are very pleasant to work with.
- bonoboTP 4mo agoYou can run a CNN and use the downsampled feature map the same way as patch tokens.
- famouswaffles 4mo ago>Transformers are more hyped and more popular with the tech enthusiasts who just read forums and news, but if you need stuff done, CNNs are still great. Vits are straight up more popular for ML research now, it's not just 'tech enthusiasts'.
- bonoboTP 4mo agoThere's a dearth of research properly comparing them.
- famouswaffles 4mo agoI'm talking about research pushing state of the art in computer vision. Vits have 100% become more popular than CNNs in most CV research.
- bonoboTP 4mo agoYes but not based on rigorous comparison. I'm not saying ViT is bad. But it took over mainly because it's the shiny new thing. It very bandwagon-Y even among PhD students.
- famouswaffles 4mo agoThere's no 'rigorous comparison' that puts CNNs over Vits in quality and Vits unlocked more use cases easier than CNNs did. That's why they're more popular, not because it's 'bandwagon-y'.
- bonoboTP 4mo agoWhat's the use case enabled vs running a ConvNeXt or EfficientNetV2 and using the resulting strided features as you would the resulting tokens of a ViT? I'm not saying that ViT is worse. Just saying that the scholarship around comparing them is very bad or nonexistent. You have to properly tune the hyperparam enters on both sides in a fair way, and use all the general modern training tricks also on the CNN side to make it fair.