4 ms·

Thanks! So I started to do that for myself, to save time and make everything reproducible. I found that when I wanted to make a small change to anything I'd ha

Thanks! So I started to do that for myself, to save time and make everything reproducible. I found that when I wanted to make a small change to anything I'd have to go back and dig up the old command lines, and sometimes I couldn't reproduce my old results. By recording the result generation in the script, including things like turning on and off the prefetchers - it made things reproducible for myself, and also encouraged experimentation since it was very easy to generate all the results after any change. Then, once you make that script for yourself, a nice side effect is that everyone else can use it too and you can skip a lot of the description about how you got your results: it's kind of a self-documenting way to explain how you got your results.

pedrocr 8y ago
That was my experience as well. Experimentation becomes much easier and for things where you want to run the same analysis every year/month/etc when more data comes out it makes things much easier. It's also great when you have computationally expensive steps to just be able to issue a build the world command at the end of the day and have the computer regenerate everything overnight. This was my version of that: https://github.com/pedrocr/codecomp https://github.com/pedrocr/codecomp Ruby made for a good way to have scripting together with more declarative build like tooks (e.g., rake). I also automated the plotting like you did by embedding R snippets. There's probably space for a good framework for this. To be the rails of scientific workflow as it were. Integrate nicely with R/LaTeX/etc for bonus points. Maybe a procrastinating PhD student somewhere will make a name for himself doing this :)