Almost everything, decision trees are the bubble sort of AI. To expand upon my comment: For most programmers Decision Trees & Nearest Neighbor, at a first glan

Almost everything, decision trees are the bubble sort of AI. To expand upon my comment: For most programmers Decision Trees & Nearest Neighbor, at a first glance look very promising and seem like natural solutions to lots of problems. However in reality one can easily prove that Decision trees tend to be suboptimal even with trivial problems due to the brittle nature of the rules represented. Also Nearest Neighbors methods face problems due to difficulty in deciding optimal space and metric (Euclidean distance is not inherently better). The reason why "Deep Learning" methods trained using non-convex optimization perform so well is because they can represent a hierarchy of features at different granularities. They can be "designed" to favor one task over the other. E.g. by choosing correct loss (triplet loss), network structure, etc. If you are interested beyond Image recognition problems, I highly recommend reading on interesting approaches such as Triplet loss for Face Recognition , Semantic Segmentation, Attention based models and Unsupervised Visual Representation Learning by Context Prediction. An interesting outcome of Deep Learning research is that a lot of papers are extremely accessible and involve little proofs/difficult math. E.g. This is a good one https://www.youtube.com/watch?v=dUgRR-JFE8s https://www.youtube.com/watch?v=dUgRR-JFE8s & http://web.cs.hacettepe.edu.tr/~aykut/classes/spring2016/bil722/slides/w08-deep-context.pdf http://web.cs.hacettepe.edu.tr/~aykut/classes/spring2016/bil...