The neural nets they compare to are remarkably shallow and thin (2 layers of 12 units each, written as "KF/DN, layers 12,12") to be described as "deep learning"

The neural nets they compare to are remarkably shallow and thin (2 layers of 12 units each, written as "KF/DN, layers 12,12") to be described as "deep learning". The surprising resurgence of deep learning since 2012 is not the observation that neural network models can learn at all -- neural nets with a handful of hidden layers were known to work fairly well since the 90s -- but that deep nets with dozens of hidden layers (enabled by GPUs) broke records and achieved state-of-the-art results on remarkably challenging tasks (famously, ImageNet 2014) in multiple important machine learning subfields (has revolutionized NLP, also AlphaGo). This paper fails to recognize why people care about deep learning as a tool for machine learning in our modern day, and by making meaningless comparisons they restrict themselves to making meaningless observations.