19 ms·
> KubernetesExecutor runs each task in an individual Kubernetes pod. Unlike CeleryCelery, it spins up worker pods on demand, hence enabling maximum usage of res
by aequitas 6y ago
> KubernetesExecutor runs each task in an individual Kubernetes pod. Unlike CeleryCelery, it spins up worker pods on demand, hence enabling maximum usage of resources.
You'll probably use up a lot of resources indeed, depending on how big your tasks are you will have quite some overhead to run each and every one in a seperate pod, compared to running them in a Celery Multiprocessing "thread" on an already running worker container.