3 ms·
This time of year, you do get the pages for things that were always broken, but nobody noticed before, because they only show up when the system has been runnin
by toast0 4y ago
This time of year, you do get the pages for things that were always broken, but nobody noticed before, because they only show up when the system has been running without changes for more than two weeks.
- purrcat259 4y agoThis actually happened to us last week in fact. No deployments revealed how a legacy background processor started losing connections to the message queue and gets stuck in a state where it never reconnects. Deployments always cycled the pods before the issue manifested.
- jldugger 4y agoThis is something a (now former) colleague of mine pointed out: that the kubernetes descheduler can enforce a maximum lifetime[0] that sort of forces continual reboots. So if your system cannot tolerate running for a long time continously, this is one method to gracefully restart long running pods. [0]: https://github.com/kubernetes-sigs/descheduler#podlifetime https://github.com/kubernetes-sigs/descheduler#podlifetime