One of the cited papers "Measuring the tendency of CNNs to Learn Surface Statistical Regularities" is also a great insight into this phenomena [0]. > Our main

One of the cited papers "Measuring the tendency of CNNs to Learn Surface Statistical Regularities" is also a great insight into this phenomena [0]. > Our main finding is that CNNs exhibit a tendency to latch onto the Fourier image statistics of the training dataset, sometimes exhibiting up to a 28% generalization gap across the various test sets. Moreover, we observe that significantly increasing the depth of a network has a very marginal impact on closing the aforementioned generalization gap. Thus we provide quantitative evidence supporting the hypothesis that deep CNNs tend to learn surface statistical regularities in the dataset rather than higher-level abstract concepts. [0] https://arxiv.org/abs/1711.11561 https://arxiv.org/abs/1711.11561