You can look at data reduction tasks as dependency networks. Generate/retrieve data, reduce data, plot data. If the intermediate artifacts are files, you can

You can look at data reduction tasks as dependency networks. Generate/retrieve data, reduce data, plot data. If the intermediate artifacts are files, you can use make to run the data pipeline. If part of the data changes, or new data arrives, you just run make again, and the whole pipeline re-does whatever is new. Very slick. It has free concurrency (make -j N). I've used this effectively several times for medium-scale (~GBs) science data processing.