4 ms·
> There's a bunch of different tools to do the same job Yup. Almost, too many tools, in fact. https://s.apache.org/existing-workflow-systems https://s.apache
by neuromantik8086 8y ago
> There's a bunch of different tools to do the same job
Yup. Almost, too many tools, in fact.
https://s.apache.org/existing-workflow-systems https://s.apache.org/existing-workflow-systems
- neuromantik8086 8y agoAs a somewhat related addendum, some of the worst pipelines I've encountered have been in scientific computing. Conceptually, DAGs are quite simple but for some reason things always end up gummed up in implementation, which is partially why scientific results are much harder to reproduce than they should be. The disconnect between how much harder pipeline creation is compared to how easy it should be in the sciences has always confused me a bit.