3 ms·
Airflow is great, honestly the biggest gotchas are passing time to operators which someone has mentioned in the thread already and setting up the initial infra
by Grimm1 6y ago
Airflow is great, honestly the biggest gotchas are passing time to operators which someone has mentioned in the thread already and setting up the initial infra is a bit annoying too. Other than that though as a batch-etl scheduler and all around job scheduler it's pretty great, it's really a very user friendly and it's graphical interface simplifies a lot of the management process. I see a lot of people prefer the non graphical libs here like Luigi or Prefect and to each there own but I really do prefer having that interface in addition to the pipelines as code line of thinking.
I also see a lot of people saying it's a solution for big companies and the like, I heavily disagree it's useful for any size company that wants to have better organization of their pipelines and provide an easy way for non technical users to check on their health.