3 ms·
Lots of people have proposed and implemented VM-based reproducibility environments. 1) it's actually harder to get reproducible computational experiments than
by dekhn 11y ago
Lots of people have proposed and implemented VM-based reproducibility environments.
1) it's actually harder to get reproducible computational experiments than they expected. For example, you can run same VM on a different processor and get different results, which makes bitwise reproduction hard, and statistical tests for nonbitwise equality are harder
2) developing and maintaining the VMs and the environment takes a fair amount of effort from skilled people
3) the resulting improvements to science don't seem to exceed the cost thresholds implied by #1 and #2, and nobody's volunteering their time.
My conclusion: good idea, but probably not critically necessary.
- jamesblonde 11y agoThat's good feedback. Although I'm considering mostly distributed systems, where full reproducability is not possible. For single-threaded, deterministic programs (discrete event simulations) where performance is not a measurement point, you can do it with VMs. The nicest thing would be to be able to parameterize experiments. Not just re-run them, but change the parameters for different runs.