That paper by David Hand is good, I had forgotten it. (Even though I presented it to a reading group at my work...). He points out a lot of ways the standard

That paper by David Hand is good, I had forgotten it. (Even though I presented it to a reading group at my work...). He points out a lot of ways the standard "supervised learning, iid feature vectors, moderate dimensionality, moderate n" problem setting is a theoretical construct that does not encompass enough of the real-world problem setting. And he highlights the fact that an epsilon improvement in error rate is not that remarkable. So, I agree that his paper was smart as a critique of a certain culture within ML practice (and publications) of totally eliminating all problem context, and just saying "Give me a bag of labeled feature vectors -- I will build a classifier with an error rate better than your classifier, and publish the result." But that is largely a straw man -- critiquing ML hackery is worthwhile and salutary, but it's going after small game. The random forest, support vector machine, and deep learning approaches should have been developed in Statistics. Why weren't they?