Also a great example of effective feature engineering. The success of gradient boosting and deep learning lies largely in the ability of those models to "learn

Also a great example of effective feature engineering. The success of gradient boosting and deep learning lies largely in the ability of those models to "learn" sophisticated high-order features from raw data. Their particular success in working with images, audio, and video lie in their ability to construct more (and more sophisticated) features, beyond what a team of smart humans with domain knowledge could have constructed by hand given huge resources. In a sense, the only difference between model architectures is the feature space that they embed the raw data into. That said, sometimes you just need to give the model a good starting place, and sometimes you can obviate A sophisticated model largely or entirely, if you can come up with highly "explanatory" features on your own. To this day, I think the ability to come up with high-impact features is one of the differentiating factors of really good industry data scientist.