2 ms·

Cargo-culting is about focusing on the process without fully understanding its purpose. Such as bureaucratic requirements for providing all data, software, para

Cargo-culting is about focusing on the process without fully understanding its purpose. Such as bureaucratic requirements for providing all data, software, parameters etc so that someone can reproduce exactly the same numbers with minimal effort. Proper reproducible research is not like that. It's about providing sufficient details that other people in the field can extrapolate the rest. That they can use similar methods with similar data to achieve similar results. Reproducing exactly the same result is not that valuable, as the "result" could be just an artifact of the specific data and specific methodology. Real validation depends on fully independent replications, with as little reuse of data and code as reasonably possible.

hytfyiv3j 3y ago
In software I have found getting the exact details and parameters very useful even if I don't intend to use them. Because when I try to do the same thing my own way and fail, then I can reference the original ones and gradually make my own version more and more similar to that, and see when it starts working. Or make their version more and more similar to mine and see when it breaks. This allows quickly easily identifying the critical difference.
zelphirkalt 3y ago
If we cannot reproduce research results, due to missing the data, parameters, or code, or whatever else, is it not cargo culting, to take that research result and blindly believe it and build on top of it? If I remember correctly, Feynman stated in the same video, that one should actually reconstruct or reproduce the experimental results we rely on.