4 ms·
> This would push the dial in the direction of incremental, very conservative research and discourage novel methods. Yes, there there are obviously edge cases,
by yokaze 6y ago
> This would push the dial in the direction of incremental, very conservative research and discourage novel methods.
Yes, there there are obviously edge cases, where reproduction is difficult or impossible. I think, the effort needed to do so should also be honoured, not simply skipped over. I disagree though, that it will stop innovation. On the contrary, if the capabilities and knowledge are spread, it allows people to build on that.
Currently, I think, we vastly favour novelty over reproducibility. We still can have "innovation", but I think, it is right to take the novelty under consideration, if it hasn't been reproduced. In my opinion, exceptionalism is used more often as not as protectionism.
If results in physics produced at the LHC are held to that standard, I do not see, why we let anyone else of the hook.
Unfortunately, in the current "publish-or-die" environment, there is next to no incentive to reproduce results or to make them reproducible. I do not get academic jobs for merely reproducing results, you cannot publish that stuff, or only on low impact papers.
And if I do produce something new and others can easily reproduce my results, then they can simply pick up the work where I started, and I lose my edge. Even if I am more open to competition, it still may be a lot of work to make it reproducible for others, time I rather can spend on furthering my research.
E.g. there is no excuse, that in computer science practically all the results can be easily reproduced, but how many are?
- jrumbut 6y agoI think, far from edge cases, there are entire, large fields that would be hampered by these guidelines. There could be parts of machine learning and the social sciences where these guidelines would work (and maybe others, I have no idea and it feels like the authors don't either). Articles like this need to get a lot more specific about what fields they're talking about. I think the best idea would be to name specific publication venues or funding opportunities that should adopt these standards. It should be that granular. Otherwise, this is one way that bureaucratic overhead, nonsensical policies, and perverse incentives are created.