5 ms·

We are using istio at scale. I have a love-hate relationship with it. It is very complex and builds on 5 other layer of abstraction (K8s, Envoy, Iptables,...).

We are using istio at scale. I have a love-hate relationship with it. It is very complex and builds on 5 other layer of abstraction (K8s, Envoy, Iptables,...). Grasping what is going on requires you to understand all of those layers first. Istio essentially adds one layer of proxy for all your ingress/egress requests and from an engineering/performance/cost perspective that is not amazing. Once it is working and deployed though it provides a solid set of functionalities as part of the infrastructure directly. AuthN/Z, mTLS, security, metrics and logs are all deployed by default without the end-user having to do anything. Eventually I expect Istio will evolve to a model that makes more sense with Ambient/eBPF (For cost/performance reasons) The community behind Istio is especially helpful and one of the main reasons why we went with this project.

slygon 3y ago
We tried Istio, but our Devops team (8 people) said they don't have the capacity to manage that complexity. We're rolling with Linkerd ever since, still a joy
datadeft 3y ago
> It is very complex and builds on 5 other layer of abstraction Yeah this is a definite no for me.
- Niksko 3y ago
  Boy do I have bad news for you about all of modern software...
  - candiddevmike 3y ago
    Not all modern software. No one is forcing anyone to use this stack.
    - amarshall 3y ago
      Yes, pretty much all modern software. The real difference is whether it’s a leaky abstraction or not. Sounds like istio is leaky.
      - bushbaba 3y ago
        Abstractions are different from automatic. Istio is closer to automatic.
ushakov 3y ago
have you tried Contour yet? https://projectcontour.io https://projectcontour.io
- crb 3y ago
  Contour is a gateway: a controller that manages Envoy proxies at the edge of a Kubernetes environment. Istio is a service mesh: a controller that manages Envoy proxies at the edge and alongside each workload. If you are using Istio, you probably don't need Contour. A year ago, a number of Envoy gateway maintainers (including Contour) announced their intention to join up to build one implementation of an Envoy gateway. They haven't made a lot of noise since, but they are apparently up to v0.4. https://blogs.vmware.com/opensource/2022/05/16/contour-and-community-build-new-envoy-gateway/ https://blogs.vmware.com/opensource/2022/05/16/contour-and-c... You can also use the Gateway API to manage Istio. So, if you are using Istio, you probably don't need Envoy Gateway either. Wherever you look, it's still Envoy. Unless of course you look at Linkerd, who have their own thing.
davewritescode 3y ago
We also run Istio at scale and feel the same way. During adopting it’s a pain, when it’s up and running it’s a dream.
- slygon 3y ago
  Did you folks try to upgrade it? For us it was a nightmare (breaking changes, etc.)
deleted 3y ago
[deleted]
JohnMakin 3y ago
“once it is working and deployed” is the caveat here. debugging issues with it at my last job was such a constant headache we nearly scrapped it for consul.
williamallthing 3y ago
You have no idea how many Istio companies have turned to Linkerd because of this very complexity. Here's one of my favorite writeups. https://nais.io/blog/posts/changing-service-mesh/ https://nais.io/blog/posts/changing-service-mesh/
- campbel 3y ago
  It's malpractice to not seriously consider Linkerd over Istio at this point.
  - strangemonad 3y ago
    What about trafaek?
    - vbezhenar 3y ago
      https://github.com/traefik/mesh https://github.com/traefik/mesh Last commit at Nov 28, 2022. In kubernetes world it means that this project is dead, I guess?
      - LilBytes 3y ago
        https://github.com/traefik/traefik https://github.com/traefik/traefik
    - 6LLvveMx2koXfwn 3y ago
      traefik? [1] 1. https://traefik.io/ https://traefik.io/
  - fosk 3y ago
    Try to give Kuma (https://kuma.io https://kuma.io) a try, also a CNCF project.
ttul 3y ago
I don’t deal with Istio daily but I observed it sucked up a vast number of hours. Mysterious cracks seem to lurk in its bowels but nobody has any idea precisely where because it’s such a complex beast. Beware.

slygon 3y ago

We tried Istio, but our Devops team (8 people) said they don't have the capacity to manage that complexity. We're rolling with Linkerd ever since, still a joy

datadeft 3y ago

> It is very complex and builds on 5 other layer of abstraction Yeah this is a definite no for me.

Niksko 3y ago
Boy do I have bad news for you about all of modern software...
- candiddevmike 3y ago
  Not all modern software. No one is forcing anyone to use this stack.
  - amarshall 3y ago
    Yes, pretty much all modern software. The real difference is whether it’s a leaky abstraction or not. Sounds like istio is leaky.
    - bushbaba 3y ago
      Abstractions are different from automatic. Istio is closer to automatic.

ushakov 3y ago

have you tried Contour yet? https://projectcontour.io https://projectcontour.io

crb 3y ago
Contour is a gateway: a controller that manages Envoy proxies at the edge of a Kubernetes environment. Istio is a service mesh: a controller that manages Envoy proxies at the edge and alongside each workload. If you are using Istio, you probably don't need Contour. A year ago, a number of Envoy gateway maintainers (including Contour) announced their intention to join up to build one implementation of an Envoy gateway. They haven't made a lot of noise since, but they are apparently up to v0.4. https://blogs.vmware.com/opensource/2022/05/16/contour-and-community-build-new-envoy-gateway/ https://blogs.vmware.com/opensource/2022/05/16/contour-and-c... You can also use the Gateway API to manage Istio. So, if you are using Istio, you probably don't need Envoy Gateway either. Wherever you look, it's still Envoy. Unless of course you look at Linkerd, who have their own thing.

davewritescode 3y ago

We also run Istio at scale and feel the same way. During adopting it’s a pain, when it’s up and running it’s a dream.

slygon 3y ago
Did you folks try to upgrade it? For us it was a nightmare (breaking changes, etc.)

deleted 3y ago

[deleted]

JohnMakin 3y ago

“once it is working and deployed” is the caveat here. debugging issues with it at my last job was such a constant headache we nearly scrapped it for consul.

williamallthing 3y ago

You have no idea how many Istio companies have turned to Linkerd because of this very complexity. Here's one of my favorite writeups. https://nais.io/blog/posts/changing-service-mesh/ https://nais.io/blog/posts/changing-service-mesh/

campbel 3y ago
It's malpractice to not seriously consider Linkerd over Istio at this point.
- strangemonad 3y ago
  What about trafaek?
  - vbezhenar 3y ago
    https://github.com/traefik/mesh https://github.com/traefik/mesh Last commit at Nov 28, 2022. In kubernetes world it means that this project is dead, I guess?
    - LilBytes 3y ago
      https://github.com/traefik/traefik https://github.com/traefik/traefik
  - 6LLvveMx2koXfwn 3y ago
    traefik? [1] 1. https://traefik.io/ https://traefik.io/
- fosk 3y ago
  Try to give Kuma (https://kuma.io https://kuma.io) a try, also a CNCF project.

ttul 3y ago

I don’t deal with Istio daily but I observed it sucked up a vast number of hours. Mysterious cracks seem to lurk in its bowels but nobody has any idea precisely where because it’s such a complex beast. Beware.