This seems so overengineered. If you're using a BI tool like Looker, you can configure how often and how long you would like your "query" to be persisted. They

This seems so overengineered. If you're using a BI tool like Looker, you can configure how often and how long you would like your "query" to be persisted. They also support dynamic triggering - when value X changes, rerun query Y. This is all 2-3 lines of LookML configuration. Also, pulling this out of the dependency graph of the rest of your ETL system seems silly. Analysts now have to schedule their queries "sometime roughly when I think the data will be available" vs "immediately once the data is available". It sounds like Chartio isn't really keeping up with their workflow. So assume for a second they need to ease this pain but can't justify a vendor switch right now. What's the fix? I think you make it easier for analysts to put jobs into Airflow. That way you aren't reinventing the wheel (building a completely isolated ETL scheduling tool) and you're actually attempting to solve one of th underlying problems (Airflow isn't "analyst friendly"). I'd love to hear other people's experience. I recognize I'm being a bit of a negative nancy.