4 ms·
When you have two systems that need to talk to each other, and one system expects the other system to be reliable, it’s difficult to completely toss out all gra
by sargun 6y ago
When you have two systems that need to talk to each other, and one system expects the other system to be reliable, it’s difficult to completely toss out all graceful shutdown and handover. Usually this logic is handled at the service discovery or load balancing layer with the deployment bits, so people don’t notice.
Kubernetes does a really good job at hiding this.
When it hides this, it also hides a couple other things with software — like failing containers, or apps that require churn or fail. I mean this is literally the reason why things like monit were built in the older days — things broke if you didn’t restart them when they broke.
In kubernetes, it also is easy to accidentally to change the system and make that behavior change (at a global level), that results in stuff like your app not being restarted as often, which results in elevated latency, and subsequent failures.
- paledot 6y agoNo system is 100% reliable, though. Some systems are close enough that we grit our teeth and pretend they are and wave away the inevitable outages that have nothing to do with our app and everything to do with an unreliable dependency. In a way, you're better off with a 99% reliable dependency than 99.9999%. If you expect it to be down some of the time, you'll plan for it in your architecture and (in theory) gracefully handle the failure whether it's due to an insufficiency warmed container, a network issue, a bad deploy, or whatever else comes along.