3 ms·
Good article! I actually like systemd, but nowadays I generally run stuff as containers, so there is less and less opportunity to use it. Here is my current se
by carlosf 7y ago
Good article! I actually like systemd, but nowadays I generally run stuff as containers, so there is less and less opportunity to use it.
Here is my current setup for Airflow:
- 1 container for the webserver
- 1 container for the scheduler
- 1 managed database (I use Postgres, it's a fairly small instance.)
- 1 S3 bucket to deploy DAGs. I mount it on my containers using s3fs-fuse.
- You can monitor the scheduler using a PID file, whereas the webserver can be monitored probing your admin URL.
- Most configuration can be done using environment variables, which is perfect for containers.
- I also configure DAG logs to be shipped to a S3 bucket.