It's true that simple models often outperform more complex ones on small datasets. But the comparison seems rather unfair in this case: the "deep learning" mode

It's true that simple models often outperform more complex ones on small datasets. But the comparison seems rather unfair in this case: the "deep learning" model employed seems to be a simple feedforward discriminative classifier, and these are known to perform badly on small datasets. There are other "deep learning" models that would likely perform much better on small sample sizes. I've written a blog post about one idea [1]. If you prefer published per-reviewed research (and you should of course) then e.g. Semi-Supervised Learning with Deep Generative Models [2] is a good starting point. 1. http://www.openias.org/hybrid-generative-discriminative http://www.openias.org/hybrid-generative-discriminative 2. https://pdfs.semanticscholar.org/b6b9/39ffc9920cd8521299a6fe9ec55775f9bf3c.pdf https://pdfs.semanticscholar.org/b6b9/39ffc9920cd8521299a6fe...