No quota here! If you like Marvin, you’ll probably like Prefect. Both are designed to be clean, Pythonic interfaces to a complex and hard-to-observe system (a d

No quota here! If you like Marvin, you’ll probably like Prefect. Both are designed to be clean, Pythonic interfaces to a complex and hard-to-observe system (a data stack; an AI stack). I think one of the key differences between Prefect and Dagster is that Prefect views orchestration as coordination, while Dagster views orchestration as reconciliation. The data stack is a complex system whose state is frequently mutated by forces outside our users’ control. Therefore, our product is focused on letting our users understand and react to those events, no matter where they come from. That could include everything from scheduling fully-orchestrated Prefect pipelines, to setting an SLA for database maintenance that Prefect doesn’t have anything else to do with. Reconciliation, in contrast, requires users to define a digital twin of their stack, in order to serve as ground truth and become the reconciliation target. Philosophically, we view Prefect as one piece of an ever-changing stack. We are focused on being as flexible as possible to fit into the stack, rather than the other way around.