4 ms·

Does anyone recommend a simpler alternative to airflow for simple multi step workflows that isolate task dependencies?

• willvarfar 6y ago
  One way to interpret the 'simpler' requirement is to suggest Luigi. Its a python framework. Its way simpler if you are a programmer. Of course a lot of people think of Airflow as 'simpler' than Luigi, because Airflow is all graphical. YMMV.
• dimberman 6y ago
  Airflow 2.0 :)
• x87678r 6y ago
  dagster
• morelandjs 6y ago
  Really happy with Prefect for smaller workflows
• eobropta 6y ago
  I recommend checking out Conducto. It handles dependencies well by leveraging Docker and it is very easy to get started. It can also handle super complicated pipelines that required scaled up processing by just changing one input on the command line.