That may be. And I don't think kernels will ever be a hot topic. But part of that is that they are a very old topic (you can see Gauss referring to them in sl

That may be. And I don't think kernels will ever be a hot topic. But part of that is that they are a very old topic (you can see Gauss referring to them in slide 22 of http://math.iit.edu/~fass/590/notes/Notes590_Ch1.pdf http://math.iit.edu/~fass/590/notes/Notes590_Ch1.pdf). I think that kinda prevents funding agencies from pushing too hard for research into them, since there is a more likely quick return on investment into a new topic. And I don't begrudge them that - funding should be given to people with new ideas which have the potential to revolutionize things. I'd also mention that there are people trying to bridge the gap between how GPs work and the benefits of neural nets (http://arxiv.org/abs/1502.05700 http://arxiv.org/abs/1502.05700). Probably another thing holding them back is their high barrier to entry - you can't do much with the theory of kernels unless you know statistics and functional analysis (and of course numerical analysis and linear algebra,) which is one of the reasons I had difficulty finding research students. To draw a parallel to numerical analysis (where I am much more comfortable than machine learning,) this is why finite element methods are much more popular than, say, boundary element methods for PDEs, despite the fact that boundary element methods are better in many circumstances. But when implementing things, especially in industry, simplicity and robustness count for something and those are not strong points of kernels. But kernels haven't gone anywhere yet, and they've been fundamental to analysis since at least David Hilbert, so they are not going anywhere any time soon. Deep learning ... we'll see. Maybe it has the legs to stick around or maybe it'll be swept aside by the next hot thing. 8 years ago all the research funding was going into uncertainty quantification, 5 years ago compressed sensing was the hot idea (thanks Terry Tao,) and now deep learning is going to light the world on fire. All of those are, and should be, hotter topics than kernels, but we'll see what we're talking about in 5 years. It won't be kernels (unless something crazy happens and I become the leader of the US, in which case I have a great idea for a new reality show) but I don't know that it will be deep learning either.