4 ms·
ImageNet-trained CNNs are biased towards texture (2019)
- jasdfijsflj 7y agoStyle transfer is done using neural networks, so it should be natural that the resultant image fools a neural network classifier. On the other hand, for the filled silhouette, the classifier network was pretty close to human.
- w_t_payne 7y agoI don't think that's what's happening here.
- mbeex 7y agoQuanta article: https://www.quantamagazine.org/where-we-see-shapes-ai-sees-textures-20190701/ https://www.quantamagazine.org/where-we-see-shapes-ai-sees-t...
- DeathArrow 7y agoWhat would be a better dataset to train CNNs?
- kleiba 7y agoPro tip: read more than just the headline.
- kzrdude 7y agoTexture is area-filling, while silhouette is "just a line", so it sounds natural that texture areas would weigh more in an image? If there is both texture and contour "signal" in the labeled inputs, how to pick what get's weighted more? This is supposed to be learned by training.
- mbeex 7y agoI think, this is one of the reasons to use much more video-based training sets and these in time-correct frame order. A moving boundary is something quite significant, but these should possibly not exclusively inspected through a hole ('convolutinal')
- wodenokoto 7y agoMy favorite titled ml article [1], from 2015, seems to argue for the same conclusion - that it is textures not shapes that drives recognition in CNN [1] “Suddenly, a leopard print sofa appears”, http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.html http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.ht...
- modeless 7y ago> it is textures not shapes that drives recognition in CNN This is a misinterpretation of the paper. It's not CNNs that are biased toward textures, it's the ImageNet dataset. As they prove by training a CNN biased toward shapes, using a modified dataset.
- fxtentacle 7y agoI thought the same. Captain Obvious called a second time. Of course CNNs are still heavily biased towards texture, because nothing has fundamentally changed in the past 5 years.
- stared 7y agoThey are, and it is not that new; the interesting part is how to solve that. For image classification, this bias for textures may be passable. For image generation, it would produce atrocious results, especially for faces. To the point, that people devised some methods to overcome this, especially with progressive networks (e.g. proGAN). See https://towardsdatascience.com/progan-how-nvidia-generated-images-of-unprecedented-quality-51c98ec2cbd2 https://towardsdatascience.com/progan-how-nvidia-generated-i... for the proGAN explanation - it starts from training on 4x4 pixel images, to get the overall shape.
- angry_octet 7y agoI'm curious if stereo vision and eyepoint movement has a benefit for humans in distinguishing shape vs texture, and perhaps that could be a promising approach for ML, but are there any databases for that? There are car focused ones like KITTI, but any more generalist ones?