I work at Kaggle. In many cases where simple models win, there's some insight into the data that the winner found - engineered a new feature, or noticed a patt

I work at Kaggle. In many cases where simple models win, there's some insight into the data that the winner found - engineered a new feature, or noticed a pattern and appropriately tuned a particular method. Where those insights exist, they often overshadow any gains by super-sophisticated ML techniques.