4 ms·
> Perhaps depositing a VM with everything ready to run. The danger then is that the VM has so much undocumented complexity that if anything goes wrong, or goes
by zrm 5y ago
> Perhaps depositing a VM with everything ready to run.
The danger then is that the VM has so much undocumented complexity that if anything goes wrong, or goes "well" when it shouldn't, no one can explain why. Which also reintroduces a vector to hide nasty tricks.
- ungamedplayer 5y agoBut you can at least figure that out. If U don't have it all results are suspect.
- kergonath 5y agoResults that haven't been independently replicated are suspect. There are just too many factors that can lead an experiment to give some results that are not transferrable or not relevant. The worst aspect of this is the lack of will or funding to replicate, replicate, and replicate again all significant results that get published. Post-processed data can be altered, but a TB of raw data is meaningless as well if it hasn't been produced properly, has been obfuscated, or is weirdly formatted. Data availability is a red herring for the vast majority of the science being made right now (almost everything that does not depend on a multi-millions dollars experiment). If data availability is an end in itself, we would just have moved the goalposts and have a data quality problem instead of a reproducibility problem.
- hirako2000 5y agothis. the point of sharing reproducible steps and not the experiment itself is that it can be fully reproduced independently. not just independently verify that the result show what the paper claims.