Same is true for any sort of science. We're trying to find some sort of a story, and thus we bias ourselves towards bugs that lead to a meaningful conclusion.

Same is true for any sort of science. We're trying to find some sort of a story, and thus we bias ourselves towards bugs that lead to a meaningful conclusion. Most experiments fail, but a bug can make them look like a success. I find it especially important if you're applying supervised machine learning techniques. Far and away, the easiest sort of bug to make is mixing up training and testing and holdout data (or not doing it at all). It's such an easy bug to make because you're trying to improve the model accuracy... and a really easy way to do that is to train and test on the same data.