You can do this in Kubernetes by combining Envoy's draining support and graceful shutdown in the backend Pod (via handling SIGTERM or adding a preStop hook to d

You can do this in Kubernetes by combining Envoy's draining support and graceful shutdown in the backend Pod (via handling SIGTERM or adding a preStop hook to delay termination). The big users of Envoy+Kubernetes are mostly running their own custom ingress controllers to coordinate this across the ingress system (easier than it sounds). https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/operations/draining https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overv... Source: I helped build a system like this for a major tech company that handles many tens of billions request daily.