3 ms·

That's not my reading. I think they are saying that models which *can* over-fit the data in both theory and practice, appear not do so when there are in fact g

• magicalhippo 6y ago
  Ah hmm yes, good point. I forgot this piece from the article: We observe a steady deterioration of the generalization error as we increase the noise level. This shows that neural networks are able to capture the remaining signal in the data while at the same time fit the noisy part using brute-force. Since they score well on the test data, they must have generalized to some degree. But since they're just as good at training on random input they also have the capacity to just memorize the training data.