I accept all of what you're saying and also spend a lot of my day reading ML papers :) To boil it down a little perhaps, the point I'm making is that the incre

I accept all of what you're saying and also spend a lot of my day reading ML papers :) To boil it down a little perhaps, the point I'm making is that the incremental gain from new ideas coming out seems to be slowly decreasing, in my view - which is perfectly natural! It obviously makes sense that that would happen. Where that thought leads me though, is the idea that we may be slowly moving towards the exploitation half of the explore/exploit choice. I'm not saying greenfield ML research is gone by any means, only that I think a lot of stuff presented as such may in fact not be because in many cases it doesn't really fundamentally re-think the problem being addressed.