2 ms·
I run a journal where we publish both “real” null results and experiments were something practical went wrong, so i have a few thoughts: 1. Ideally peer review
by tefkah 2y ago
I run a journal where we publish both “real” null results and experiments were something practical went wrong, so i have a few thoughts:
1. Ideally peer review would catch this. A badly setup study should be critiqued in peer review. Forcing scientists to first publish their methods before doing the experiment also helps, as it validates the experimental setup before hand.
I also think it’s worth publishing studies where a null result was reached due to some error in experimental setup or other factors, as long as it’s presented as such and reflected upon. This can still be valuable information for future experiments. Offering scientists social capital for that (an “official” publication, citations) might also incentivize scientists to publish the results as is, rather than making it appear as a “true” null result, or even as a non-null one (eg through p hacking).
2. While obviously possible, given the amount of effort scientists have to go through to raise funding for an experiment nowadays, i find it highly unlikely that people would go through this effort.
3. This is already possible and a problem. This is a problem of academic misconduct, has very little to do with null results.
The current publishing system is of course already set up to be gamed, so I understand your worries. But null results should be published, as they are just science. Even if someone were to “game” the system by publishing a ton of null results, those publications should be held to the same level of scrutiny as any other publication. If someone is extremely prolific in replicating existing studies and comes up with a ton of null results, that should be lauded and those papers should be published, no?
I do believe the entire idea of a researchers output only being recognized by being allowed to be published in a journal is terrible and should be abolished, but baby steps I guess.