2 ms·

I will check out the docker operator. I didn't want to drop down into the bash executor to run docker containers because it felt a little dirty. My dev team i

I will check out the docker operator. I didn't want to drop down into the bash executor to run docker containers because it felt a little dirty. My dev team is excited to use airflow but, in my estimation, keeping them all in lockstep with a single requirements.txt seems a little far fetched. ;)

tedmiston 9y ago
It definitely feels weird at first but it's also worked well for us running millions of dockerized tasks. :) Yeah, I can see where you're coming from there. Some people will run multiple separate Airflow instances as another alternative solution.