5 ms·
There's a bunch of disciplines (live animal experimentation, microorganisms and organic chemistry come to mind, though they aren't "my" fields) where it's genui
by PeterisP 6y ago
There's a bunch of disciplines (live animal experimentation, microorganisms and organic chemistry come to mind, though they aren't "my" fields) where it's genuinely difficult to perform experiments, where it's reasonably common for experimenter to screw up the experiment in multiple ways, and failure to replicate may just as well indicate not a flaw with the original experimenter but a weakness of the skills of the team trying to replicate.
One could argue that it's a failure of not sufficiently detailed descriptions of the experiment, but it is how it is in different disciplines. A relevant example for that is a bunch of earlier machine learning research (current methods subjectively seem more robust in this regard) some of which were very difficult to replicate from scratch because it was very finicky and relied on many tiny details that simply can't all fit in the couple pages of a standard paper, but it was definitely not nonsense, because it could be replicated if you knew all the best practices from previous experience. I mean, providing code for a ML paper doesn't change whether the research results there are nonsense or not, but many people might have a very hard time replicating that research without the code.