3 ms·
> Second, you almost certainly don't want to use kube-proxy. It uses (abuses?) iptables/nftables […] every major cloud provider ([…] Azure […]) …yeah… the IP "
by deathanatos 4y ago
> Second, you almost certainly don't want to use kube-proxy. It uses (abuses?) iptables/nftables […] every major cloud provider ([…] Azure […])
…yeah… the IP "pool" for Services on Azure does come from a pool of NAT IPs, but it's still an imaginary pool? (It's not a real IP in the cloud's network — which is also more NAT IPs.) AFAICT, it's not different than how the article describes. kube-proxy runs in the cluster, even. (This is w/ the Azure CNI.)
In fact, the Service IP pool is so imaginary, if you accidentally allocate the same CIDR in the real Vnet, AKS gets all sorts of mad at you, and withholds upgrades until you repent. Support, similarly, won't do anything, no matter how obviously unrelated your actual problem is.
Contrast to GCP's GKE: service IPs show up in a VPC, IIRC as "secondary" IPs for a subnet. That does happen closer to how I think you're describing it. (GKE's model is night and day better. I hate finding out that, somehow, somewhere, in our past, someone created an overlapping network in Azure. It's a PITA to deal with, every time it happens.)
(Also, how much I'd love to jettison everything that has anything to do with NAT for IPv6.)
Also, IIRC, if you debug an AKS node, you'll see the iptables "abuse" you mention. And yes, it makes me cry tears of blood. I think IPVS is supposed to straighten some of that out and make me cry less tears of blood?
> provision mTLS certificates to pods that let them authenticate themselves as a given service identity.
You can do this with core-dns being your resolver, and using the normal unadorned service names. (CoreDNS really doesn't matter here.) It just changes what name you need to stick in the subjectAltNames. You'll need to "be your own CA" for this, of course. But cert-manager makes that pretty easy to do. (No more difficult than getting a cert from a real CA, and if anything … potentially easier.)
I've not had a problem with CoreDNS? Ours seems to cope fine at about 500 requests / min. The bigger problem is stuff in the cluster being really irresponsible about how many DNS queries they make…
- jmillikin 4y agoThank you for the correction regarding Azure! I knew they had a CNI plugin, but did't realize it has the limitations you describe (I've only used GCP and AWS). > I've not had a problem with CoreDNS? Ours seems to cope fine at about > 500 requests / min. The bigger problem is stuff in the cluster being > really irresponsible about how many DNS queries they make… My expectation for DNS is that name lookups should function at thousands of requests per second. Serving that kind of QPS implies caching, and thus some sort of update notification -- for example watching the EndpointSlices resource for changes, and pushing them directly into the DNS server's state. (As you might notice from my article + employment history, my expectations around cluster sizes differs from the community norm.)