3 ms·
Would love to know the pain points so we can make it better - Composer, Airflow, or both.
by chimerasaurus 7y ago
Would love to know the pain points so we can make it better - Composer, Airflow, or both.
- DevKoala 7y agoTriggering a DAG externally fails consistently for 1% of requests, and 100% of the time when the web server API goes down, which is too often (502). I understand that in the realm of Airflow, triggering a DAG externally is perhaps a bit of an anti-pattern, but it should not have been documented then: https://cloud.google.com/composer/docs/how-to/using/triggering-with-gcf https://cloud.google.com/composer/docs/how-to/using/triggeri... I have cases open with the composer team, and I am upgrading the cluster again this weekend, but I have lost hope at this point; I am designing a solution to stop using external triggers.