5 ms·

Mizu – API traffic viewer for Kubernetes

132 points by kiyanwang 5y ago

dolni 5y ago
Neat idea and a cool project. That something like this is necessary is yet another testament to the complexity of Kubernetes.
- eddieroger 5y ago
  Is it necessary? I've operated in Kubernetes environments for several years and not had a visualizer like this and survived just fine. If anything, it's neat you can do this in k8s relatively easily and "for free" on an existing cluster. The only complexity this calls out is that of microservice architecture done to an extreme, but not that of Kubernetes.
  - dolni 5y ago
    > Is it necessary? I'm assuming that they didn't spend time authoring this for no good reason. It got 2700 stars on GitHub, which isn't nothing. > The only complexity this calls out is that of microservice architecture done to an extreme, but not that of Kubernetes. There are better approaches to dealing with tracking requests through a microservice architecture than a ham-fisted packet capture. By default, the way networking is configured in Kubernetes is exceptionally complex. It certainly stands to reason that a packet capture may be useful at some point.
  - cpitman 5y ago
    At many of my clients, when an app breaks on kubernetes, it's generally on the admins to show that it wasn't the cluster's fault before the app team tales ownership of the issue. Tools that help very clearly show that traffic made it to the app, and what the app responded with, are super helpful.
- throwanem 5y ago
  In the sense that the need for Wireshark is a testament to the complexity of IP, I suppose, sure.
  - dolni 5y ago
    Yes, but we're not talking about IP and this misses the point. Kubernetes, as a technology, has support tooling being built around every interface it has. Just to install Kubernetes you've got Kubespray, Kops, kubeadm, minikube, or you can choose from a smattering of managed Kubernetes options. For performance you've got cAdvisor, and you probably need to export those metrics somewhere like Prometheus/Grafana. Then there's Helm, which is yet another abstraction over deployments to Kubernetes, which is SUPPOSED to be the thing Kubernetes simplifies. And let's not forget about Rancher. Can't run one Kubernetes cluster, because security and other reasons. So let's simplify running ten of them. I can't think of a single other technology that was built to ostensibly simplify things, only to be completely enshrouded by support tooling.
    - throwanem 5y ago
      In full production deployment, sure, but in full production deployment everything is "enshrouded by support tooling" - or had better be, if you want to be able to recognize and respond to incidents in a timely fashion. But if that's not a requirement, there are easier options - my Kubernetes home lab, for example, I set up in about 20 minutes with k3s [1], and it's been ticking happily along for over a year at this point with zero administrative burden. Granted, it'll be a PITA if that ever breaks, but it's a lab and I don't have to care more about downtime than I want to; at worst I'll tear it down, follow my notes from last time to stand it up again, and reapply my manifests out of source control. Meanwhile, as a developer, I find that k8s does simplify a great deal with regard to deploying services and managing them once deployed - this, in turn, makes life easier for our infra folks, who no longer have to spend so much time splitting focus to provide direct dev support for deployment issues. Which is after all one of the reasons to run Kubernetes at all! It's built to simplify things, but not to simplify everything; there is a deal of irreducible complexity involved in the domain k8s addresses, and because it is indeed irreducible, that complexity can at best be managed and abstracted, not removed. That's what k8s intends to do, and in my experience it does that quite well - not without tradeoffs, true enough, but when in our line of work is that ever not the case? [1] I mention this in particular because I used to take a view much like you do here, until an HN commenter pointed out in response to my own such fussing that it didn't have to be as complicated as I had understood it to be, and recommended trying k3s in that wise. It's been a year and I no longer remember who that was, but they were absolutely correct regardless!
boomskats 5y ago
This is a brilliant and ambitious project. However last time I tried it a few months ago, it seemed to drop a lot of traffic once the throughput became anything non-trivial (100+ requests per second across 50ish microservices). Maybe it's a question of UX, more likely it's a question of user error (and therefore partly UX). In any case, I can't wait for it to get better!
- xur17 5y ago
  My original view was that this wold be used for development and staging environments - is the idea that this can be used in a production environment as well?
j4ah4n 5y ago
Neat project and will plan to kick the tires. I like the workflow and especially the cleanup aspect for a tool such as this.
remram 5y ago
How does it work? The docs don't seem to mention.
- ljloisnflwef 5y ago
  Welcome to Kubernetes! I don't know what went wrong but the ecosystem prefers software presented as a black box and shrouded with mystery. If you contrast this with Envoy, it has stellar documentation -- Envoy wants you to know how it works! Envoy is used and abused by Istio (k8s wrapper) which has terrible documentation, history of horrific upgrade/update story, (& software quality).
  - remram 5y ago
    That hasn't really been my experience, I have found an abundance of documentation (sometimes getting lost down the rabbit hole). But I have kept clear of the corporate buzzword solutions, that might be why.
- thanos999 5y ago
  Istio delivers encrypted packets across services/ pods but in the pod, envoy proxy does the decryption and puts the packet unencrypted back on the interface to be consumed by the application. In kubernetes, for calico CNI, the pod networking is done using veth pair. One interface remains on the host to be hooked to the host bridge and the other interface is sent to the pod namespace. Mizu is deployed as a daemon set(one pod per node) and with enough privilege you can "tcpdump" the pod interface(the one where the envoy proxy puts back the packets unencrypted) (similar to "ip netns exec") Basically, they tap the pod interface from host level. (They have a blog post about this as well: https://up9.com/gain-visibility-into-istio-mtls-traffic-with-mizu https://up9.com/gain-visibility-into-istio-mtls-traffic-with...) This is my understanding so I might be wrong
  - rednerrus 5y ago
    How much latency is this going to add?
dbrowne 5y ago
Looks interesting.
xur17 5y ago
Just installed this in a dev cluster, and I'm ecstatic. Does a lot of what I've been looking for, including: * log of all traffic * service graph * automatic swagger doc generation THANK YOU!!
atlaschen 5y ago
actually used Mizu to quickly validate I dont have Spring4Shell in my dev/pre-prod, super cool product!
jumperabg 5y ago
Absolute miracle. This is a must-have tool.

132 points by kiyanwang 5y ago

Neat idea and a cool project. That something like this is necessary is yet another testament to the complexity of Kubernetes.

eddieroger 5y ago
Is it necessary? I've operated in Kubernetes environments for several years and not had a visualizer like this and survived just fine. If anything, it's neat you can do this in k8s relatively easily and "for free" on an existing cluster. The only complexity this calls out is that of microservice architecture done to an extreme, but not that of Kubernetes.
- dolni 5y ago
  > Is it necessary? I'm assuming that they didn't spend time authoring this for no good reason. It got 2700 stars on GitHub, which isn't nothing. > The only complexity this calls out is that of microservice architecture done to an extreme, but not that of Kubernetes. There are better approaches to dealing with tracking requests through a microservice architecture than a ham-fisted packet capture. By default, the way networking is configured in Kubernetes is exceptionally complex. It certainly stands to reason that a packet capture may be useful at some point.
- cpitman 5y ago
  At many of my clients, when an app breaks on kubernetes, it's generally on the admins to show that it wasn't the cluster's fault before the app team tales ownership of the issue. Tools that help very clearly show that traffic made it to the app, and what the app responded with, are super helpful.
throwanem 5y ago
In the sense that the need for Wireshark is a testament to the complexity of IP, I suppose, sure.
- dolni 5y ago
  Yes, but we're not talking about IP and this misses the point. Kubernetes, as a technology, has support tooling being built around every interface it has. Just to install Kubernetes you've got Kubespray, Kops, kubeadm, minikube, or you can choose from a smattering of managed Kubernetes options. For performance you've got cAdvisor, and you probably need to export those metrics somewhere like Prometheus/Grafana. Then there's Helm, which is yet another abstraction over deployments to Kubernetes, which is SUPPOSED to be the thing Kubernetes simplifies. And let's not forget about Rancher. Can't run one Kubernetes cluster, because security and other reasons. So let's simplify running ten of them. I can't think of a single other technology that was built to ostensibly simplify things, only to be completely enshrouded by support tooling.
  - throwanem 5y ago
    In full production deployment, sure, but in full production deployment everything is "enshrouded by support tooling" - or had better be, if you want to be able to recognize and respond to incidents in a timely fashion. But if that's not a requirement, there are easier options - my Kubernetes home lab, for example, I set up in about 20 minutes with k3s [1], and it's been ticking happily along for over a year at this point with zero administrative burden. Granted, it'll be a PITA if that ever breaks, but it's a lab and I don't have to care more about downtime than I want to; at worst I'll tear it down, follow my notes from last time to stand it up again, and reapply my manifests out of source control. Meanwhile, as a developer, I find that k8s does simplify a great deal with regard to deploying services and managing them once deployed - this, in turn, makes life easier for our infra folks, who no longer have to spend so much time splitting focus to provide direct dev support for deployment issues. Which is after all one of the reasons to run Kubernetes at all! It's built to simplify things, but not to simplify everything; there is a deal of irreducible complexity involved in the domain k8s addresses, and because it is indeed irreducible, that complexity can at best be managed and abstracted, not removed. That's what k8s intends to do, and in my experience it does that quite well - not without tradeoffs, true enough, but when in our line of work is that ever not the case? [1] I mention this in particular because I used to take a view much like you do here, until an HN commenter pointed out in response to my own such fussing that it didn't have to be as complicated as I had understood it to be, and recommended trying k3s in that wise. It's been a year and I no longer remember who that was, but they were absolutely correct regardless!

boomskats 5y ago

This is a brilliant and ambitious project. However last time I tried it a few months ago, it seemed to drop a lot of traffic once the throughput became anything non-trivial (100+ requests per second across 50ish microservices). Maybe it's a question of UX, more likely it's a question of user error (and therefore partly UX). In any case, I can't wait for it to get better!

xur17 5y ago
My original view was that this wold be used for development and staging environments - is the idea that this can be used in a production environment as well?

j4ah4n 5y ago

Neat project and will plan to kick the tires. I like the workflow and especially the cleanup aspect for a tool such as this.

remram 5y ago

How does it work? The docs don't seem to mention.

ljloisnflwef 5y ago
Welcome to Kubernetes! I don't know what went wrong but the ecosystem prefers software presented as a black box and shrouded with mystery. If you contrast this with Envoy, it has stellar documentation -- Envoy wants you to know how it works! Envoy is used and abused by Istio (k8s wrapper) which has terrible documentation, history of horrific upgrade/update story, (& software quality).
- remram 5y ago
  That hasn't really been my experience, I have found an abundance of documentation (sometimes getting lost down the rabbit hole). But I have kept clear of the corporate buzzword solutions, that might be why.
thanos999 5y ago
Istio delivers encrypted packets across services/ pods but in the pod, envoy proxy does the decryption and puts the packet unencrypted back on the interface to be consumed by the application. In kubernetes, for calico CNI, the pod networking is done using veth pair. One interface remains on the host to be hooked to the host bridge and the other interface is sent to the pod namespace. Mizu is deployed as a daemon set(one pod per node) and with enough privilege you can "tcpdump" the pod interface(the one where the envoy proxy puts back the packets unencrypted) (similar to "ip netns exec") Basically, they tap the pod interface from host level. (They have a blog post about this as well: https://up9.com/gain-visibility-into-istio-mtls-traffic-with-mizu https://up9.com/gain-visibility-into-istio-mtls-traffic-with...) This is my understanding so I might be wrong
- rednerrus 5y ago
  How much latency is this going to add?

dbrowne 5y ago

Looks interesting.

xur17 5y ago

Just installed this in a dev cluster, and I'm ecstatic. Does a lot of what I've been looking for, including: * log of all traffic * service graph * automatic swagger doc generation THANK YOU!!

atlaschen 5y ago

actually used Mizu to quickly validate I dont have Spring4Shell in my dev/pre-prod, super cool product!

jumperabg 5y ago

Absolute miracle. This is a must-have tool.