3 ms·
>"... note that iptables has a big performance hit when you start getting to hundreds to thousands of overlay IPs and you’ll notice it if you manage a mid-to-la
by bogomipz 8y ago
>"... note that iptables has a big performance hit when you start getting to hundreds to thousands of overlay IPs and you’ll notice it if you manage a mid-to-large sized cluster"
Thanks for your reply. Could you elaborate on this performance issue? Since both iptables and IPVS are both in the kernel I would curious to hear why the issue doesn't exist between IPVS and overlays.
- sisk 8y agoStarting to fall outside of my domain so I encourage correction however one of the reasons, from what I understand, is that iptables resolves rules sequentially (top to bottom within the chain) versus IPVS which is a hash table lookup—the more rules, the slower iptables gets. The way that the loadbalancing works with the iptables implementation is rules with decreasing probability. That means that, worse case, the kernel with match and fall through one rule per replica. In other words, with 20 pods that match a service selector, 20 rules will be processed before the packet destination is rewritten to the right pod IP and port. Not a big impact when you’re dealing with a few rules but a tangible impact when you’re dealing with the amount of rules created by a good-sized kubernetes cluster.
- bogomipz 8y agoThanks for the detailed explanation. Yes in larger environments with thousands of iptables rules, using ipsets(hashing and constant time lookups) gets around this bottleneck) you are describing. I wonder if ipsets is not an option with Kube-proxy and iptables though?