2 ms·
Yes and no, but first we need to define novelty. I'm in ML and all the academic issues are on full display here (comes with any hyped research area). Reviewer
by godelski 4y ago
Yes and no, but first we need to define novelty.
I'm in ML and all the academic issues are on full display here (comes with any hyped research area). Reviewer 2 will always knock you on novelty no matter what you do. That's because all research is to some extent derivative. As it should be. We shouldn't be reinventing the wheel. But on the other hand, all research is incremental (especially in a publish or perish system. You can't publish breakthroughs frequently). Leaps and bounds don't happen often and they generally do either by accident (someone doing something dumb) or because someone was spending a lot of time on that area (likely not publishing in main track or at all). But novelty is also hard to measure post hoc. Most things are non-obvious a priori but are obvious post hoc. Reviewers go after these all the time and all it encourages is researchers to make their works convoluted. I've literally been told that papers I've written are too simple and straight forward. I've also seen those papers get better reviews after complexifying the text. I don't feel good about this btw.
But I want to give a perfect example of all this in ML. Diffusion models. The stochastic noise process has been around for decades. People have been applying them to generative models for awhile too. But they were basically unknown until they beat GANs at image generation and within a few short months they are a household name (even more than GANs). This looks like a bigger breakthrough than it is because they were flying under the radar for so long and really just one team spent a lot of time with them AND had access to A LOT of compute resources. But still, I think this should make you think about how novelty works and if you're reviewing, maybe don't squash papers because you think they are dead ends. Diffusion was a "dead end" until it wasn't.