4 ms·
dependency management. if task #3 fails, any task depending on it shouldn't run. not easy to do with cron based triggers
by bosie 6y ago
dependency management. if task #3 fails, any task depending on it shouldn't run. not easy to do with cron based triggers
- kfk 6y agoYes but that is a feature of parallel and Dask does it well.
- bosie 6y agoI don't know too much about dask, how would you build a node in a Dask DAG to execute a java app and analyse its results (i.e. say database entries) to evalutate the success of that step?
- pgwhalen 6y agoAirflow is generally not aware of data flow, it just runs something once its upstream dependencies have finished. Less useful in a way, but also much more broadly applicable.