3 ms·
The reasons were related more to accessibility and the data team's ability to fold the orchestration framework into their workflow and not be constrained by it.
by computershit 4y ago
The reasons were related more to accessibility and the data team's ability to fold the orchestration framework into their workflow and not be constrained by it. A lot of that was on me not having the time to make it easy to adopt, but Prefect just offered immediate adoption (being able to shell out, run notebooks, arbitrary Docker containers or k8s pods, in addition to a very unobtrusive decorating pattern) that was too great to pass up.
What Dagster has going for it in this space is pragmatism. It really nails all of the problem points of data ops (with resources & sensors specifically). If I was consulting for a shop that needed data pipelines and they had good eng, I'd recommend Dagster in a heartbeat.