12 ms·

What exactly is a workflow engine? At a previous job, we had a fair amount of celery tasks and logic around starting them based on user input or on a schedule,

What exactly is a workflow engine? At a previous job, we had a fair amount of celery tasks and logic around starting them based on user input or on a schedule, retrying on failures and marking progress or cleaning up state in various databases. Is that a workflow engine?

mrfox321 4y ago
Sure. Open source analogue would be Apache Airflow. Abstractly, it's some directed acrylic graph (DAG) that is asynchronously computed, sometimes on a schedule. Unfortunately, most things fall under DAG. But the framework / engine exists to manage the complexity of the ever-extending pipelines declared by the engineers Event/push-based workflows also fall under this taxonomy.
- froh 4y ago
  why acyclic? review steps can send stuff back to previous input steps, can't they?
  - thesnide 4y ago
    It is usually better to have explicit returns.
  - mrfox321 4y ago
    Yeah, I mean, the computation can be re-materialized. DAG may too specific. It's really a dependency graph, that likely has a DAG topology.