I agree there are many options in this space. Two others to consider: - https://airflow.apache.org/ https://airflow.apache.org/ - https://github.com/spotify/l

I agree there are many options in this space. Two others to consider: - https://airflow.apache.org/ https://airflow.apache.org/ - https://github.com/spotify/luigi https://github.com/spotify/luigi There are also many Kubernetes based options out there. For the specific use case you specified, you might even consider a plain old Makefile and incrond if you expect these all to run on a single host and be triggered by a new file showing up in a directory… I like Airflow because you can give access to the web UI to operators and they can kick/run/stop tasks or graphs of tasks. Both Airflow and Luigi expect you to express your workflow as a DAG in Python code.