4 ms·
Dagster is extremely nice to work with. I did a bakeoff of Prefect vs Dagster internally at my current employer, and while we ended up going with Prefect for re
by computershit 4y ago
Dagster is extremely nice to work with. I did a bakeoff of Prefect vs Dagster internally at my current employer, and while we ended up going with Prefect for reasons, I am still so impressed with the way Dagster approaches certain pain points in the orchestration of data pipelines and its solution for them.
- theptip 4y ago> for reasons I'd love to hear more on this. I've not evaluated Prefect, and am currently keeping an eye on Dagster. What trade-offs does Prefect win?
- 64StarFox64 4y agoI did a baby bakeoff internally in my prior role ~18mo ago now. Prefect felt nicer to write code in but perhaps not as easy to find answers in the docs (though their Slack is phenomenal). Ended up going with Prefect so I could focus on biz/ETL logic with less boilerplate, but I'm sure Dagster is not a bad choice either. Curious to hear about parent's experience
- computershit 4y agoThe reasons were related more to accessibility and the data team's ability to fold the orchestration framework into their workflow and not be constrained by it. A lot of that was on me not having the time to make it easy to adopt, but Prefect just offered immediate adoption (being able to shell out, run notebooks, arbitrary Docker containers or k8s pods, in addition to a very unobtrusive decorating pattern) that was too great to pass up. What Dagster has going for it in this space is pragmatism. It really nails all of the problem points of data ops (with resources & sensors specifically). If I was consulting for a shop that needed data pipelines and they had good eng, I'd recommend Dagster in a heartbeat.