Overfitting is about reading too much in a properly sampled data set from any distribution. It comes about when your model has so many parameters it can be made

Overfitting is about reading too much in a properly sampled data set from any distribution. It comes about when your model has so many parameters it can be made to fit anything. Regularization keeps it in check, but it remains an issue. The example you cite has an issue earlier in the chain. Here, we are dealing with a biased sample from some distrubution. The results from that won't generalize to the unbiased distrubution, no matter how great your model is.