4 ms·
There is a difference between reproducibility and repeatability. Reproduction is an independent experiment producing commensurate results. Repetition is the sam
by reitblatt 15y ago
There is a difference between reproducibility and repeatability. Reproduction is an independent experiment producing commensurate results. Repetition is the same lab repeating the experiment and finding the same results. Sharing code actually reduces the independence of experiments. Worse, sharing buggy code introduces systematic errors across "independent experiments". Scientists already deal with similar issues due to a small number of vendors of various tools, but software is pretty different. Systematic measuring biases can be detected and calibrated, but software bugs rarely lend themselves to such corrections. Because science depends upon independent reproducibility and NOT repeatability, there's an argument to be made that blindly sharing code is actually detrimental to scientific reproducibility.
The real question we should be asking is whether opening and sharing these code bases will result in an increase in quality that offsets the loss of experimental independence.