You mean tasks like computing the rank of a matrix? (http://oneweirdkerneltrick.com/rank.pdf http://oneweirdkerneltrick.com/rank.pdf) ;) Image super-resolution

You mean tasks like computing the rank of a matrix? (http://oneweirdkerneltrick.com/rank.pdf http://oneweirdkerneltrick.com/rank.pdf) ;) Image super-resolution is an ill-posed inverse problem: many possible high-resolution images would be reduced to the same low-resolution image. However, some of the possibilities are more realistic than others. Consider a grey pixel in a photograph from real life. It's more likely that it was downsampled from a 2x2 block of grey pixels than from a 2x2 black-and-white checkerboard pattern. We apply such knowledge by adding regularization to the problem, or using a prior distribution in a Bayesian formulation. Deep learning is very good at memorizing those priors by looking at real world data. I think, in problems where the prior is important and complicated, data-driven approaches have a big advantage over hand-engineered approaches. That does not mean they will take over every kind of problem.