TLDR: Complexity. The deprecation lifecycle, and running ingress controllers in an automatic scaling group. The first isn't as much of an issue if you have a

TLDR: Complexity. The deprecation lifecycle, and running ingress controllers in an automatic scaling group. The first isn't as much of an issue if you have a (partially) dedicated team for managing your clusters, but can be prohibitively expensive (effort / time-wise) for smaller organisations. The second highlights a bigger problem in K8s in general. I'll have to give a little background first: If you run an Nginx ingress controller on a node that's part of an ASG — i.e. a group where nodes can disappear, or increase in number — you will experience service disruption to a small percentage of your requests, every time a scaling event occurs. This is caused by a misalignment between timeout values for your load balancer and Nginx, which can not be fixed: * https://github.com/kubernetes/ingress-nginx/issues/6281 https://github.com/kubernetes/ingress-nginx/issues/6281 * https://github.com/kubernetes/ingress-nginx/issues/6791 https://github.com/kubernetes/ingress-nginx/issues/6791 * https://github.com/kubernetes/ingress-nginx/issues/7175 https://github.com/kubernetes/ingress-nginx/issues/7175 The fix is to only run the controllers on nodes that reside in a separate statically sized group, and perform updates to them out of hours when necessary :| I'll leave you to decide on whether that's a fix or not, but the larger point it highlights is how _theoretically_ everything's great in K8s, but the headaches introduced by the complexity often make it not worth it. Another example is pod disruption budgets. These are needed because the behaviour of K8s when instructed to shutdown a node is, well, to shutdown the node. Seems reasonable, until you realise that it doesn't handle moving the workloads off that node _first_. No, at some point later, the scheduler realises the pods aren't running and schedules them somewhere else. So you use a combination of PDBs to tell K8s that it must keep n pods of this deployment running at all times, and distribution rules to tell it pods must run on different nodes. This solution falls apart when you have pods that should only have a single instance running.