From paper: Tabular datasets are the last "unconquered castle" for deep learning, with traditional ML methods like Gradient-Boosted Decision Trees still perform

From paper: Tabular datasets are the last "unconquered castle" for deep learning, with traditional ML methods like Gradient-Boosted Decision Trees still performing strongly even against recent specialized neural architectures. My but this statement seems more than a little grandiose. Never mind that XGBoost still does well on a substantial portion of ML challenges (supposedly). The bigger problematic is that there's a confusion of maps and territories in this way of talking of machine learning. The field of ML has made a certain level of palpable progress by having created a number of challenges and benchmarks and then doing well on them. But success on a benchmark isn't necessarily the same as a success the "task" broadly. An NLP test doesn't imply mastering real language, a driving benchmark doesn't imply master over the road driving. etc. Notably, success on a benchmark also "isn't nothing". In a situation like the game of go, the possibilities can be fully captured "in the lab" and success at tests indeed became success against humans. But with driving or language, things are much more complicated. What I would say is that benchmark success seems to produce at least a situation where the machine can achieve human-like performance for some neighborhood (or tile or etc) limited in time, space and subject. Of course, driving is the poster-child for the limitations of "works most of the time" but lot of "intelligent" activities require an ongoing thread of "reasonableness" aside from having an immediate logic. Anyway, it would be nice if our ML folks looked at this stuff more as a beginning than as a situation where they're poised on success.