3 ms·
What alternative do you recommend?
by ssahoo 2y ago
What alternative do you recommend?
- annexrichmond 2y agoI've done a lot evaluation of such frameworks, and I hope to publish more on it. It really depends on your requirements. I would look at Prefect, Flyte, Dagster, and Temporal ahead of Airflow, though.
- 8organicbits 2y agoWhere do you think you'd publish it? I'm interested to read the details.
- whalesalad 2y agoWe've been running Dagster for ~6 months on ECS and it has been rock solid. (knocks on wood)
- 0cf8612b2e1e 2y agoAirflow is kind of the default platform. If you do not have a goto alternative which is superior in multiple dimensions today, I think that says the problem itself is hard. Might as well go with the devil everyone knows.
- computershit 2y agoPrefect has more polish and is easier to get started than any of the existing options. We've been running their self-hosted for over three years and it basically stays out of the way. We looked at Dagster as well as Airflow. I really, really liked Dagster but the BI team didn't. I cannot imagine using Airflow for anything meaningful and respecting myself at the end of a work day. The local development experience was abysmal. Deployments sucked. That being said, if you're not using anything except maybe cron right now, and if you don't care about the solution being a proper data pipelines orchestration platform trademark symbol, I'd recommend starting with Windmill.
- liveoneggs 2y agojenkins