3 ms·

Sure. But for approximately the same level of effort, you could generate a new paper in the same template. It would likely be easier to get that "novel result"

by shimon 9y ago

Sure. But for approximately the same level of effort, you could generate a new paper in the same template. It would likely be easier to get that "novel result" published than the refutation.

beagle3 9y ago
And that's generally the truth in every scientific field. Right now you have a "reproduction renaissance" in psychology and cancer, where you can get published, and get acknowledged, if you tear down a high level paper. But in general, unless you aim at refuting the highest level, it is hard to get published and advances your career nowhere, so people have no incentive to do it.
- nonbel 9y ago
  A non-replication does not "tear down" a paper... It is not any kind of "attack", your attitude is one of the sources of the problem. Edit: When I did medical research I wished people would try to replicate my findings so I could see how consistent the results were and how well my model fit future data... In fact it was really annoying that it was unlikely anyone would ever try to do a direct replication.
  - elviejo 9y ago
    That is the problem: Why wouldn't try to do a replication? Because the incentives are to provide "new research", not to confirm or deny a known result. It seem to me that most probably no-one has gotten tenure by reproducing, disproving results.
  - beagle3 9y ago
    My attitude? What attitude is that? A single non-replication does not. Consistent failure to replicate, after communicating with the original author, does. See e.g. http://www.slate.com/articles/health_and_science/cover_story/2016/03/ego_depletion_an_influential_theory_in_psychology_may_have_just_been_debunked.html http://www.slate.com/articles/health_and_science/cover_story... http://www.nature.com/nature/journal/v483/n7391/full/483531a.html http://www.nature.com/nature/journal/v483/n7391/full/483531a... Or just google "reproducibility crisis", the two most common subjects that come up are cancer research and psychology research.

by shimon 9y ago

Sure. But for approximately the same level of effort, you could generate a new paper in the same template. It would likely be easier to get that "novel result" published than the refutation.

beagle3 9y ago

And that's generally the truth in every scientific field. Right now you have a "reproduction renaissance" in psychology and cancer, where you can get published, and get acknowledged, if you tear down a high level paper. But in general, unless you aim at refuting the highest level, it is hard to get published and advances your career nowhere, so people have no incentive to do it.

nonbel 9y ago
A non-replication does not "tear down" a paper... It is not any kind of "attack", your attitude is one of the sources of the problem. Edit: When I did medical research I wished people would try to replicate my findings so I could see how consistent the results were and how well my model fit future data... In fact it was really annoying that it was unlikely anyone would ever try to do a direct replication.
- elviejo 9y ago
  That is the problem: Why wouldn't try to do a replication? Because the incentives are to provide "new research", not to confirm or deny a known result. It seem to me that most probably no-one has gotten tenure by reproducing, disproving results.
- beagle3 9y ago
  My attitude? What attitude is that? A single non-replication does not. Consistent failure to replicate, after communicating with the original author, does. See e.g. http://www.slate.com/articles/health_and_science/cover_story/2016/03/ego_depletion_an_influential_theory_in_psychology_may_have_just_been_debunked.html http://www.slate.com/articles/health_and_science/cover_story... http://www.nature.com/nature/journal/v483/n7391/full/483531a.html http://www.nature.com/nature/journal/v483/n7391/full/483531a... Or just google "reproducibility crisis", the two most common subjects that come up are cancer research and psychology research.