1. Managed, Cloud Composer. Cloud Composer is getting there. It feels much less buggy then just 8 months ago when I started using it, and it is improving rather

1. Managed, Cloud Composer. Cloud Composer is getting there. It feels much less buggy then just 8 months ago when I started using it, and it is improving rather quickly. One downside with Composer, though, is that it must be run in its own GKE cluster, and it deploys the Airflow UI to App Engine. These two things can make it a bit of a pain to use alongside infrastructure deployed into another GKE cluster if you need the two to interact. I would probably still recommend Composer over deploying your own Airflow into GKE, as having it managed is nice. 2. Freedom. For some tasks we run containers in GKE, for other we use things like the PythonOperator or PostgresOperator. A note here: Using containers with Airflow is not trivial. In addition to needing some CI process to manage image building/deployment, having the ability to develop and test DAGs locally takes some extra work. I would only recommend it if you are already invested in containers and are willing to devote the time to ops to get it all working. 3. X-com is useful for small amounts of data, like if one task needs to pass a file path, IDs, or other parameters to a downstream task. For everything else have a task write its output to something like S3 or a database that another task will read from. All in all, I would say use Airflow if you need the visibility and dependency management. Don't use it if you could get away with something like cron and some scripts or a simple pool of celery workers. Also, don't use it if your workflows are highly dynamic. For example, if you have a situation where you need to run a task to get a list of things, then span x downstream tasks based on the contents of the list. Airflow wants the shape of the DAG to be defined before it is run. Hope that helps.