3 ms·
Depends on how you've configured airflow. It polls its dag directory constantly, and may do some db queries for each poll (don't quote me on this, I definitely
by habitue 7y ago
Depends on how you've configured airflow. It polls its dag directory constantly, and may do some db queries for each poll (don't quote me on this, I definitely don't know everything about the internals of airflow). From what I remember you can reduce the polling period in config
- quantguy11959 7y agoYeah I’ll need to do some research, if I can only poll the db for just the time I need this will be a huge cost saving for me. I think the db only gets queried if a dag actually runs but I’m not 100% sure.