This article "50 Years of Data Science" by David Donoho (2017) had a sharp point about Kaggle-like competitions being under-valued by stats: "To my mind, the

This article "50 Years of Data Science" by David Donoho (2017) had a sharp point about Kaggle-like competitions being under-valued by stats: "To my mind, the crucial but unappreciated methodology driving predictive modeling’s success is what computational linguist Mark Liberman (Liberman 2010) has called the Common Task Framework (CTF). An instance of the CTF has these ingredients: (a) A publicly available training dataset involving, for each observation, a list of (possibly many) feature measure- ments, and a class label for that observation. (b) A set of enrolled competitors whose common task is to infer a class prediction rule from the training data. (c) A scoring referee,to which competitors can submit their prediction rule. The referee runs the prediction rule against a testing dataset, which is sequestered behind a Chinese wall. The referee objectively and automatically reports the score (prediction accuracy) achieved by the submitted rule. ... The general experience with CTF was summarized by Liberman as follows: 1. Error rates decline by a fixed percentage each year, to an asymptote depending on task and data quality. 2. Progress usually comes from many small improvements; a change of 1% can be a reason to break out the champagne. 3. Shared data plays a crucial role—and is reused in unex- pected ways. ... The author believes that the Common Task Framework is the single idea from machine learning and data science that is most lacking attention in today’s statistical training. https://www.tandfonline.com/doi/full/10.1080/10618600.2017.1384734 https://www.tandfonline.com/doi/full/10.1080/10618600.2017.1...