3 ms·
> Pods are removed from the Service/Endpoint in async They list this in the Takeaways but I don't see that they took any action to work around this problem. SI
by aeyes 4y ago
> Pods are removed from the Service/Endpoint in async
They list this in the Takeaways but I don't see that they took any action to work around this problem. SIGTERM to the pod and updating the service/endpoint race and you will absolutely see traffic getting sent to terminated processes because of it.
The only "solution" I know is to put a sleep in the preStop hook to delay the SIGTERM so that the pod can be removed from the service. Cloud native!
- jayd16 4y agoSeems like they were not seeing the race, and instead only saw that open connections would stay open if not explicitly closed during the graceful shutdown. Once the graceful shutdown was properly executed, it closed any open connections to that pod and stopped the 502s they were seeing. Sounds like either the race wasn't happening or they didn't see/care about it.