3 ms·

One potential approach is Apache Airflow.

• tomrod 7y ago
  The Airflow creators went on to make Prefect, I hear.
  • kfk 7y ago
    Do you have a personal contact? You seem to also be interested in similar things as me. By the way if you are ok with a bit of custom code you can do a lot of what airflow/prefect do with dask.delayed
    • tomrod 7y ago
      I'm a big fan of dask, though haven't used it in a deployment/scheduling contact. I'd love to connect.
  • BerislavLopac 7y ago
    I have worked with Airflow extensively, but this is the first time I heard about Prefect, and mind blown! Looking at the docs, it seems like they have resolved most of the things we had to work hard around in Airflow -- I definitely need to look more deeply into it. Thank you!