3 ms·
That's where I see recomputation as not quite pushing the same goals as reproducibility, even though its advocates often couch them as the same goal. Recomputat
by _delirium 13y ago
That's where I see recomputation as not quite pushing the same goals as reproducibility, even though its advocates often couch them as the same goal. Recomputation can be useful, but re-running the exact same code in the same virtual machine isn't really an independent reproduction of the claimed scientific result. That often benefits from not using the original source code; two independently written implementations claiming to implement the same approach and achieving the same results is a much better reproduction.
In the natural sciences, independent reproduction often finds subtle dependencies on the original apparatus that change the interpretation: when lab B tries to reproduce lab A's results on slightly different equipment and can't, it can highlight an unnoticed dependency on some specific feature of the original equipment, and may throw into question the original paper's conclusions. You would never have found that if, instead of lab B independently reproducing the result, lab A just packed up their equipment into a shipping container and shipped it to lab B, who unpacked and ran it unchanged. That's what the VM approach is arguing for, and that's not really reproducibility.
- turingfan 13y agoI completely agree. A recomputation of an experiment is not ensuring reproducibility of the scientific result. It's ensuring reproducibility of the individual experiment. The analogy I have given is with cold fusion. If we could reproduce their exact lab setup then we could find out if the results were real - i.e. were not misread or anything, and assuming they were, have a chance of explaining the anomaly. But no, it's not the same as reproducibility.