3 ms·
Theres a push in the bioinformatics world to try to use these (WDL in particular). The main use-case is to write complex data analysis pipelines in WDL so they
by mbreese 11y ago
Theres a push in the bioinformatics world to try to use these (WDL in particular). The main use-case is to write complex data analysis pipelines in WDL so they can be shared and executed on different environments (local machines, cloud VMs, etc...).
Think of it more like an "execution engine" rather than a workflow engine. You define multiple jobs, their resources, and how they interact. The engine executes each step.
Unfortunately though, to me, it seems clunky to use. It's not something that I'd like to write by hand. It is logically very similar to defining Java builds with Ant XML files, if that helps. You can get the job done, but if you want to have any sort of complexity or logic, it's not going to be easy. Any XML/JSON/YAML format is going to have this issue.
I think the jury is still out though as to how much it gets used outside of a few large academic centers. I see it as mainly helpful for large-scale analysis projects where you have to manage cloud resources. Not many people have that problem. I'm personally only planning on using it as an output for my own language (which is basically a Make-like format that submits dynamic jobs to HPC clusters). I have yet to see how WDL can work with dynamic workflows (where you might have more than one path to an output or change the workflow based on a command like argument). It might catch on, but I've seen this show before.