7 ms·

Some thoughts: - This implementation is a good example of why Kubernetes ingress has been in beta for 2 years ;-). Ingress doesn't do very much, so the Kong te

by rdli 8y ago

Some thoughts: - This implementation is a good example of why Kubernetes ingress has been in beta for 2 years ;-). Ingress doesn't do very much, so the Kong team has had to implement a custom set of extensions to Ingress ("KongIngress") so you get functionality such as rate limiting, etc. This blog post https://blog.getambassador.io/kubernetes-ingress-nodeport-load-balancers-and-ingress-controllers-6e29f1c44f2d https://blog.getambassador.io/kubernetes-ingress-nodeport-lo... covers some of the nuances of this. - I wonder how the REST APIs of Kong are reconciled with the declarative config. Seems like this could be complicated. - IIRC, all of the NGINX ingress controllers take the route of custom annotations on the ingress object vs the KongIngress approach. I wonder what people will prefer. - The post alludes to intelligent connection draining with Kong but doesn't give any details. I'm particularly curious because I thought Kong was based on NGINX so wondering how Kong does it given the underlying engine.

captn3m0 8y ago
I totally agree. We've been using Traefik+Ingress and the lack of customizability in the configuration has been a minor pain point. Coming from nginx/apache rulesets where you can pretty much route a request however you want to a simple Ingress is jarring.
lobster_johnson 8y ago
Do we know what the Kubernetes' team's official plan for fixing the ingress situation is? My opinion is that ingress portability isn't really that useful in practice. You want implementation lock-in at the ingress level, since the implementation matters. If I need to target platforms in a generic way (e.g. make something that works on both AWS and GKE), I can use Helm or some other templating solution to generate the right manifests with the right settings. So "KongIngress" isn't a problem for me, and rather than struggle with today's situation on GKE, I'd much rather have a "GoogleIngress" with all the GLBC bells and whistles (well, there aren't many, but at least timeouts and a CDN toggle!) and not care about it being portable.
- rdli 8y ago
  At KubeCon Austin in December there was a lot of conversation about it. Some want portability, some want expressiveness, everyone agrees that current ingress isn't expressive enough. In practice, everyone adds custom annotations one way or another (whether it's NGINX ingress, or Ambassador, or KongIngress). So at some level, the notion of Kube ingress is a moot point, since everyone has forked it :-).
  - smarterclayton 8y ago
    I kind of agree. At this point it may just be better to write controllers that read generic policy from a different object and the base definition from the Kube object. But at that point, the extra ingress object is just a convenience.
- alxvio 8y ago
  As a frequent user of ingress, I agree that portability isn't useful in practice and that requirement is a hindrance on ingress evolution. A lot of work has been put into CRDs, yet parts of the community push for a "portable" ingress. With all the one-off features/settings, users will inevitably make substantial changes to their ingress object - might as well use CRDs with clear specs. The alternative is annotation hell or associated configmaps. I've heard some say they want to be able to list out all external addresses exposed from the cluster, but that can be solved with a new resource more akin to endpoints.
- jkarneges 8y ago
  > CDN toggle Hmm, what would this do exactly?
  - lobster_johnson 8y ago
    I'm referring to Cloud CDN [1], which is per-backend setting that is disabled by default. Unlike some other offerings (like AWS CloudFront), the CDN is transparent and built into the GLBC. Just by turning on the CDN, your load balancer automatically gets a globally distributed cache. [1] https://cloud.google.com/cdn/ https://cloud.google.com/cdn/
meddlepal 8y ago
The reason we didn't adopt Kubernetes Ingresses for Ambassador (https://getambassador.io https://getambassador.io) is more or less the same reason you cite in your first bullet. The interface is too simple and while it may work for the simple case of configuring a reverse proxy it rapidly falls apart in the face of things like advanced API capabilities such as rate limiting, shadow traffic, canaries etc. It turns out these are things people really care about.
SEJeff 8y ago
This also requires a datastore such as Postgres or Cassandra instead of just using the kubernetes api via a CRD or something else. Makes it a bit of a downer to use.
- rdli 8y ago
  +1. Traefik & Ambassador & the NGINX ingress controllers can persist to Kubernetes, and I think that's a big win for simplicity.

by rdli 8y ago

captn3m0 8y ago

I totally agree. We've been using Traefik+Ingress and the lack of customizability in the configuration has been a minor pain point. Coming from nginx/apache rulesets where you can pretty much route a request however you want to a simple Ingress is jarring.

lobster_johnson 8y ago

Do we know what the Kubernetes' team's official plan for fixing the ingress situation is? My opinion is that ingress portability isn't really that useful in practice. You want implementation lock-in at the ingress level, since the implementation matters. If I need to target platforms in a generic way (e.g. make something that works on both AWS and GKE), I can use Helm or some other templating solution to generate the right manifests with the right settings. So "KongIngress" isn't a problem for me, and rather than struggle with today's situation on GKE, I'd much rather have a "GoogleIngress" with all the GLBC bells and whistles (well, there aren't many, but at least timeouts and a CDN toggle!) and not care about it being portable.

rdli 8y ago
At KubeCon Austin in December there was a lot of conversation about it. Some want portability, some want expressiveness, everyone agrees that current ingress isn't expressive enough. In practice, everyone adds custom annotations one way or another (whether it's NGINX ingress, or Ambassador, or KongIngress). So at some level, the notion of Kube ingress is a moot point, since everyone has forked it :-).
- smarterclayton 8y ago
  I kind of agree. At this point it may just be better to write controllers that read generic policy from a different object and the base definition from the Kube object. But at that point, the extra ingress object is just a convenience.
alxvio 8y ago
As a frequent user of ingress, I agree that portability isn't useful in practice and that requirement is a hindrance on ingress evolution. A lot of work has been put into CRDs, yet parts of the community push for a "portable" ingress. With all the one-off features/settings, users will inevitably make substantial changes to their ingress object - might as well use CRDs with clear specs. The alternative is annotation hell or associated configmaps. I've heard some say they want to be able to list out all external addresses exposed from the cluster, but that can be solved with a new resource more akin to endpoints.
jkarneges 8y ago
> CDN toggle Hmm, what would this do exactly?
- lobster_johnson 8y ago
  I'm referring to Cloud CDN [1], which is per-backend setting that is disabled by default. Unlike some other offerings (like AWS CloudFront), the CDN is transparent and built into the GLBC. Just by turning on the CDN, your load balancer automatically gets a globally distributed cache. [1] https://cloud.google.com/cdn/ https://cloud.google.com/cdn/

meddlepal 8y ago

The reason we didn't adopt Kubernetes Ingresses for Ambassador (https://getambassador.io https://getambassador.io) is more or less the same reason you cite in your first bullet. The interface is too simple and while it may work for the simple case of configuring a reverse proxy it rapidly falls apart in the face of things like advanced API capabilities such as rate limiting, shadow traffic, canaries etc. It turns out these are things people really care about.

SEJeff 8y ago

This also requires a datastore such as Postgres or Cassandra instead of just using the kubernetes api via a CRD or something else. Makes it a bit of a downer to use.

rdli 8y ago
+1. Traefik & Ambassador & the NGINX ingress controllers can persist to Kubernetes, and I think that's a big win for simplicity.