4 ms·
First, although you are correct that the endpoint is removed from the proxies, the removal is asynchronous and so the proxy configurations may not be up to date
by otterley 5y ago
First, although you are correct that the endpoint is removed from the proxies, the removal is asynchronous and so the proxy configurations may not be up to date before the shutdown is initiated. Also, more and more load balancers are bypassing node ports these days, preferring instead to route traffic directly to pods, and so it’s becoming a less effective mechanism.
Second, the challenge is that app developers are rarely aware of the need, let alone how to write SIGTERM handlers that finish handling existing requests before exiting. For them it’s yet another cognitive and testing burden that is usually out of their business expertise domain, and to my knowledge few of the common web service frameworks implement a reliable shutdown handler out of the box.
On the other hand, if the orchestrator simply waited for the LB to signal that the pod was removed from the backend list before initiating the shutdown process, then it would be more robust by design.
- bogomipz 5y ago>"Also, more and more load balancers are bypassing node ports these days, preferring instead to route traffic directly to pods, and so it’s becoming a less effective mechanism" Could you or someone else say which LBs are currently bypassing node ports? Also how does a LB actually bypass a nodeport since a service type of Loadbalancer in Kubernetes always uses a nodeport? How exactly does an external Loadbalancer connect to a pod without first connecting to a port on the physical host?
- otterley 5y agoThe AWS load balancer controller for Elastic Load Balancers is capable of configuring routing directly to pods in an EKS cluster because those pods get a routable VPC subnet IP address by default. They’re not stuck behind bridges connected to non-routable subnets, so no NAT needed.
- krab 5y agoI can agree with that. :-)