4 ms·
I am wondering why they still use the celery executor while the kubernetes executor is the go-to one for large deployments. I have used the celery executor and
by higeorge13 4y ago
I am wondering why they still use the celery executor while the kubernetes executor is the go-to one for large deployments. I have used the celery executor and had so many issues and stuck tasks in the past and frequently fine-tune the celery configuration in airflow config.
- kmbfjr 4y agoIn practice it should be better, but it introduces its own awfulness. It requires retuning the scheduler loops, if there needs to be a scaling increase, the tasks run the risk of timing out.