2 ms·

They're complementary - LoadBalancer Services are L3, Ingresses are L4+. More often than not in-cluster Ingress providers (eg. nginx-ingress-controller) will in

by q3k 7y ago

They're complementary - LoadBalancer Services are L3, Ingresses are L4+. More often than not in-cluster Ingress providers (eg. nginx-ingress-controller) will in fact use LoadBalancers to actually route L3 traffic into the cluster in the first place. Without being able to create LoadBalancer services there's no easy way to get any traffic into your cluster other than using NodePorts, and these have tons of shortcomings.

pepemon 7y ago
NodePort is a bad practice for a such case, you are right. You get additional unnecessary routing on CNI level between the cluster nodes just to get traffic into the Ingress controller pod, for example. The solution here is to use hostNetwork mode with the pool of workers solely dedicated to scheduling Ingress controllers. Traffic directly hits NGINX/Envoy/HAProxy/Traefik/whatever and gets into the cluster without additional intermediates. You need a load balancing solution for this pool of Ingress controllers, that's right, but as I said before, this setup gives you the flexibility to cook load balancing as you desire. BTW, community NGINX Ingress controller is able to ingress L3 (TCP) traffic into the cluster.
- q3k 7y ago
  > You need a load balancing solution for this pool of Ingress controllers, that's right, but as I said before, this setup gives you the flexibility to cook load balancing as you desire. Sure, but this means that you cannot use LoadBalancers, which is painful. It means every payload has to be configured both at k8s level and then externally. That somewhat defeats the use of k8s as a self-service platform internally in an organization (other dev/ops teams need to go through a centralized ops channel to get traffic ingressing into the cluster if for some reason they can't use an Ingress). > BTW, community NGINX Ingress controller is able to ingress L3 (TCP) traffic into the cluster. Yes, but it's configurable via a single ConfigMap (which limits self-service if you're running a multi-tenant orga-wide cluster, unless you bring your own automation), and still you only have one namespace of ports, ie. a single external address for all ports - unless you complicate your 'external' LB system even further. With all these caveats, I really don't understand why not just run metallb.

by q3k 7y ago

pepemon 7y ago

NodePort is a bad practice for a such case, you are right. You get additional unnecessary routing on CNI level between the cluster nodes just to get traffic into the Ingress controller pod, for example. The solution here is to use hostNetwork mode with the pool of workers solely dedicated to scheduling Ingress controllers. Traffic directly hits NGINX/Envoy/HAProxy/Traefik/whatever and gets into the cluster without additional intermediates. You need a load balancing solution for this pool of Ingress controllers, that's right, but as I said before, this setup gives you the flexibility to cook load balancing as you desire. BTW, community NGINX Ingress controller is able to ingress L3 (TCP) traffic into the cluster.

q3k 7y ago
> You need a load balancing solution for this pool of Ingress controllers, that's right, but as I said before, this setup gives you the flexibility to cook load balancing as you desire. Sure, but this means that you cannot use LoadBalancers, which is painful. It means every payload has to be configured both at k8s level and then externally. That somewhat defeats the use of k8s as a self-service platform internally in an organization (other dev/ops teams need to go through a centralized ops channel to get traffic ingressing into the cluster if for some reason they can't use an Ingress). > BTW, community NGINX Ingress controller is able to ingress L3 (TCP) traffic into the cluster. Yes, but it's configurable via a single ConfigMap (which limits self-service if you're running a multi-tenant orga-wide cluster, unless you bring your own automation), and still you only have one namespace of ports, ie. a single external address for all ports - unless you complicate your 'external' LB system even further. With all these caveats, I really don't understand why not just run metallb.