3 ms·
Redundancy helps with uncorrelated failures, but if you start getting requests that cause memory pressure and OOM kills, you could lose the entire cluster. I ag
by erik_seaberg 7y ago
Redundancy helps with uncorrelated failures, but if you start getting requests that cause memory pressure and OOM kills, you could lose the entire cluster. I agree that swap is often too slow to help, but "self-destruct and hope the rest of the cluster is healthy right now" is not a good failure mode. Don't overcommit in prod.
- mason55 7y agoSure, I agree with that 100%. I guess if you had a scenario where you were operating near your limits and something happened that caused one or a few hosts to start swapping, and because they were up the LB kept sending them traffic instead of redirecting it to other hosts, then you could avoid a thundering herd-type problem. But that still relies on uncorrelated failures. Otherwise that seems like an orthogonal discussion to swap vs. OOM killer. It's something to be aware of and you need circuit breakers in place but I don't think it's an argument for letting hosts swap instead of having the OOM killer deal with it.