3 ms·
Yeah, -j just about does it, though with Snakemake you can also specify other resources (memory, I/o, etc) for each task, and it will keep all of them under quo
by rflrob 9y ago
Yeah, -j just about does it, though with Snakemake you can also specify other resources (memory, I/o, etc) for each task, and it will keep all of them under quota. So if you tell snakemake that task A takes six cores, but B-K take only one each, it will solve the knapsack problem to come up with some kind of optimal order for them, and on your 8-core machine you could have A, B, and C running, or B-K.
For a while, I muddled through with Make and a custom cluster job-submitter script, but the real Snakemake killer feature for me was multiple, independent wildcards. So I can, in one rule, have dir/{databasename}/{query}.tsv and rapidly hit multiple databases at once. If you are very clever (and rule one of Make is don’t be clever), you can have a GNUMake wildcard get parsed into multiple wildcards with the wimpy pattern substitution rules, but this is not a fun way to relax.
- jschwartzi 9y agoOh, cool, so I can finally run make -j target without worrying about the LMK in Linux killing my window manager!