4 ms·

We use cert-manager to get letsencrypt certs in GKE, and automatically push them to the Google global load balacer's. https://github.com/jetstack/cert-manager

We use cert-manager to get letsencrypt certs in GKE, and automatically push them to the Google global load balacer's. https://github.com/jetstack/cert-manager https://github.com/jetstack/cert-manager

atombender 8y ago
Doesn't that still require that you create a maximum of 10 host rules per ingress? That makes it harder to automate things. With the current system, we can use one consistent IP or CNAME for all new domains. With the 10-certs-per-GLB limitation, we'd have to manage the DNS accordingly so that all the domains are correctly spread out among the N different ingress GLBs. At the time we started with Kubernetes, cert-manager was listed as alpha quality and not recommended for production. Even today, it seems Gandi (which we use for DNS) still isn't supported; at least it's not in the documented list [1]. LEGO supports Gandi, so I'm not sure if maybe the documentation is wrong here. [1] https://docs.cert-manager.io/en/latest/reference/issuers/acme/dns01/ https://docs.cert-manager.io/en/latest/reference/issuers/acm...
- JMTQp8lwXL 8y ago
  I don't think this limitation applies when using Kubernetes with cert-manager. When you create Kubernetes Service API objects of type LoadBalancer, you get (by default) get a TCP load balancer on GCP. SSL termination becomes the responsibility of the cluster. The certificate (and private key) is stored inside the cluster, too, via secrets. To have cert-manager automatically create and renew certificates, all you have to do is update your ingress host/tls YAML configuration. Using cert-manager, you can continue exposing your app under many domain names with a single IP (via an A record) or abstractly through a CNAME record. I still haven't added more than 10 domains to manually verify your concern, however. But Kubernetes isn't making any changes to my load balancer as I've added domains. I would re-visit to see if cert-manager is right for your use case: I noticed no mention of it being alpha quality on the README. However, they do point out it is 0.x and there could be breaking changes to the API later.
  - atombender 8y ago
    So the method you describe ends up using a TCP GLB, but the point of using an HTTP GLB is to enjoy all the benefits of the GLB. With a HTTP GLB, you get a very cheap distributed, effectively global CDN that doesn't require special configuration. You also get features like health checking, logging (which you can pipe to BigQuery), Google's "just works out of the box" edge caching (negating the need for an external caching CDN like Fastly), and so on. My understanding is that the typical use case is to run cert-manager together with ingresses, where cert-manager's will allocae certs and create Kubernetes secrets for each. If you wire up ingresses with those secrets, and you use the "gce" or default ingress class, then you end up getting TLS termination at the GLB level, very easily. However, because of the 10-certs-per-LB, you run into the problem I described before. There's no way around it except to create ingresses that contain a maximum of 10 hosts. (It's a hard limit, not a quota; you can't petition to have the limit increased.) If you have 60 domains, that necessarily means 6 GLBs, and 6 different IPs to point DNS to. We're using Traefik today as a custom ingress controller with a TCP GLB. So we get one external IP, and Traefik handles TLS termination (via Let's Encrypt). So this is effectively the same as running cert-manager plus something like the Nginx ingress controller.
    - JMTQp8lwXL 8y ago
      Yeah, I should have clarified in my case I am using the nginx ingress controller, but Traefik would also suffice.

We use cert-manager to get letsencrypt certs in GKE, and automatically push them to the Google global load balacer's. https://github.com/jetstack/cert-manager https://github.com/jetstack/cert-manager

atombender 8y ago

Doesn't that still require that you create a maximum of 10 host rules per ingress? That makes it harder to automate things. With the current system, we can use one consistent IP or CNAME for all new domains. With the 10-certs-per-GLB limitation, we'd have to manage the DNS accordingly so that all the domains are correctly spread out among the N different ingress GLBs. At the time we started with Kubernetes, cert-manager was listed as alpha quality and not recommended for production. Even today, it seems Gandi (which we use for DNS) still isn't supported; at least it's not in the documented list [1]. LEGO supports Gandi, so I'm not sure if maybe the documentation is wrong here. [1] https://docs.cert-manager.io/en/latest/reference/issuers/acme/dns01/ https://docs.cert-manager.io/en/latest/reference/issuers/acm...

JMTQp8lwXL 8y ago
I don't think this limitation applies when using Kubernetes with cert-manager. When you create Kubernetes Service API objects of type LoadBalancer, you get (by default) get a TCP load balancer on GCP. SSL termination becomes the responsibility of the cluster. The certificate (and private key) is stored inside the cluster, too, via secrets. To have cert-manager automatically create and renew certificates, all you have to do is update your ingress host/tls YAML configuration. Using cert-manager, you can continue exposing your app under many domain names with a single IP (via an A record) or abstractly through a CNAME record. I still haven't added more than 10 domains to manually verify your concern, however. But Kubernetes isn't making any changes to my load balancer as I've added domains. I would re-visit to see if cert-manager is right for your use case: I noticed no mention of it being alpha quality on the README. However, they do point out it is 0.x and there could be breaking changes to the API later.
- atombender 8y ago
  So the method you describe ends up using a TCP GLB, but the point of using an HTTP GLB is to enjoy all the benefits of the GLB. With a HTTP GLB, you get a very cheap distributed, effectively global CDN that doesn't require special configuration. You also get features like health checking, logging (which you can pipe to BigQuery), Google's "just works out of the box" edge caching (negating the need for an external caching CDN like Fastly), and so on. My understanding is that the typical use case is to run cert-manager together with ingresses, where cert-manager's will allocae certs and create Kubernetes secrets for each. If you wire up ingresses with those secrets, and you use the "gce" or default ingress class, then you end up getting TLS termination at the GLB level, very easily. However, because of the 10-certs-per-LB, you run into the problem I described before. There's no way around it except to create ingresses that contain a maximum of 10 hosts. (It's a hard limit, not a quota; you can't petition to have the limit increased.) If you have 60 domains, that necessarily means 6 GLBs, and 6 different IPs to point DNS to. We're using Traefik today as a custom ingress controller with a TCP GLB. So we get one external IP, and Traefik handles TLS termination (via Let's Encrypt). So this is effectively the same as running cert-manager plus something like the Nginx ingress controller.
  - JMTQp8lwXL 8y ago
    Yeah, I should have clarified in my case I am using the nginx ingress controller, but Traefik would also suffice.