Yes, that's extremely important. I've had great success, but replacing Airflow, Luigi, and friends with a cron-ed Makefile target refreshing some database table

Yes, that's extremely important. I've had great success, but replacing Airflow, Luigi, and friends with a cron-ed Makefile target refreshing some database tables (usually Materialized views). I've then used this tool to visualize the execution graph. https://github.com/lindenb/makefile2graph https://github.com/lindenb/makefile2graph The result looks like this: https://tselai.com/data/graph.png https://tselai.com/data/graph.png The convenient thing is that each node in the execution graph is in a different environment. Some are shell scripts, a some are Python scripts while others are SQL queries.