This article is a little too glib in my opinion, preferring citations and statements to substance and explanations. For a more cutting and insightful critique,

This article is a little too glib in my opinion, preferring citations and statements to substance and explanations. For a more cutting and insightful critique, watch Ali Rahimi's short talk at NIPS 2017 (where he was presenting for a paper that won the "Test of time" award, for standing out in value a decade after publication). The standing ovation he received at the end indicate that his comments resonated with a significant fraction of the attendees. https://www.youtube.com/watch?v=Qi1Yry33TQE&feature=youtu.be&t=660 https://www.youtube.com/watch?v=Qi1Yry33TQE&feature=youtu.be... Here's a teaser from the talk: "How many of you have devised a deep neural net from scratch, architecture and all, and trained it from the ground up, and when it didn't work, felt bad about yourself, like you did something wrong? This happens to me about every three months, and let me tell you, I don't think it's you [...] I think it's gradient descent's fault. I'll illustrate..." [Addendum] Ben Recht and Ali Rahimi published an adendum to the talk, elaborating on the direction they envision -- http://www.argmin.net/2017/12/11/alchemy-addendum/ http://www.argmin.net/2017/12/11/alchemy-addendum/ Ali also has a post taking a stab at organizing some puzzling basic observations about deep learning, and motivating that with analogous historical progress in optics -- http://www.argmin.net/2018/01/25/optics/ http://www.argmin.net/2018/01/25/optics/ ----- PS: The first 11 minutes, on the idea of using random features (the main idea in the research he presented) are also interesting.