3 ms·
I understand the concern. It's kind of like using a web framework — you probably wouldn't write your own form processing and ORM code to avoid injection attack
by tedmiston 8y ago
I understand the concern. It's kind of like using a web framework — you probably wouldn't write your own form processing and ORM code to avoid injection attacks when it's already been built. Airflow has analogous features and protections for tasks/workflows.
Most companies adopting Airflow already have workflow requirements like this, even if it's just a single transform or moving data from one system to another.
Even if you have just a two-step workflow with Task B dependent upon success of Task A, Airflow offers protection, historical stats, email alerting, etc over trying to schedule successive cron jobs with built-in assumptions, hacking together a dependency system, etc.
To me, Airflow is the Honda of this domain. Overall it's a relatively small and simple framework from the DAG author's perspective.