Yup, it is very frustrating - and discouraging. There are some tools designed for replication - Snakemake, for one, in python, which seems to work pretty well.

Yup, it is very frustrating - and discouraging. There are some tools designed for replication - Snakemake, for one, in python, which seems to work pretty well. But, I think it adds unnecessary dependencies and complexity. I support using basic using basic UNIX tools for reproducibility, when possible: git to make all code publicly accessible, and make to reproduce all experiments and figures. I've implemented it in 2 comp bio projects (one which relies on publicly accessible data, and one which doesn't). In both cases it massively improved my workflow. It has the added benefit of being language-independent: when I have to blend R and python, it works seamlessly. I first started thinking about this problem after I heard this talk by Arfon Smith [1]. It contextualizes reproducibility tangentially in discussions about viewing modern research as a heterogeneous (nodes can represent papers, experiments, software, authors, etc.) dependency network. [1] https://speakerdeck.com/arfon/academias-biggest-blind-spot https://speakerdeck.com/arfon/academias-biggest-blind-spot