You could try the tool my group builds, Arvados (https://arvados.org https://arvados.org). We use Common Workflow Language (CWL) as the workflow language. Arvad

You could try the tool my group builds, Arvados (https://arvados.org https://arvados.org). We use Common Workflow Language (CWL) as the workflow language. Arvados works great for very large computations and data management at petabyte scale. It really shines in a production environment where data provenance is key. Intelligent handling of failures (which are inevitable at scale) is a key part of the Arvados design.