4 ms·

I'm curious how managable this in practice. On the one hand, Linkerd brings some real technical solutions to the table that Kubernetes can't solve right now —

I'm curious how managable this in practice. On the one hand, Linkerd brings some real technical solutions to the table that Kubernetes can't solve right now — K8s' L4 load-balancing can only load-balance entire connections, not requests. For that reason, K8s can't do anything to assist you with timeouts, retries, rate-limiting, tracing or metrics collection. I like the idea of a semi-transparent middleman that handles these things for you and lets your applications be small and simple. On the other, this setup replaces a core K8s concept with its own, less proven components. As I understand it, Linkerd's daemon replaces service lookup entirely; it will talk to the API server to get service endpoints, which means it needs to invent its own logic to keep up to date with changing pod configurations. It had better be solid stuff! To reduce surface area, I wonder if running one Linkerd per pod in a sidecar container is a viable option, or would that be crazy? As an aside, can anyone explain why the example needs to start "kubectl proxy" in a sidecar container? Kubernetes already has a perfectly good way for apps to talk to the API server through a service account [1]. Seems iffy to me. [1] http://kubernetes.io/docs/user-guide/accessing-the-cluster/#accessing-the-api-from-a-pod http://kubernetes.io/docs/user-guide/accessing-the-cluster/#...

adlleong 10y ago
Running one linkerd per pod as a sidecar is definitely not crazy. You can see an example of this configuration here: https://linkerd.io/getting-started/k8s/ https://linkerd.io/getting-started/k8s/ Using kubectl proxy takes care of locating and sending the service account token and certificates. This is described a bit farther down in the Kubernetes doc that you linked.
williamallthing 10y ago
Great point about the need for production worthiness. FWIW the load balancing pool management, etc., in linkerd is straight from Finagle, which is probably more proven than Kubernetes, if anything (powers Twitter, Pinterest, SoundCloud, etc.) Some K8s-specific bits are fresher code, but even those have been tested at non-trivial scale, e.g. https://monzo.com/blog/2016/09/19/building-a-modern-bank-backend/ https://monzo.com/blog/2016/09/19/building-a-modern-bank-bac...
theptip 10y ago
Is the complexity of "invent[ing] its own logic to keep up to date with changing pod configurations" really that substantial? Each Service maps to an Endpoints set, which is a list of pod IP:port tuples. Seems straightforward to watch Services, and update your mesh targets when something changes. Obviously not trivial, but not a huge API surface. And the k8s API is architected to make such reimplementation as easy as possible, by composing the kube-proxy from publicly available cluster APIs. Seems like quite a thin layer of new glue to connect two moderately-well-proven components together.

adlleong 10y ago

Running one linkerd per pod as a sidecar is definitely not crazy. You can see an example of this configuration here: https://linkerd.io/getting-started/k8s/ https://linkerd.io/getting-started/k8s/ Using kubectl proxy takes care of locating and sending the service account token and certificates. This is described a bit farther down in the Kubernetes doc that you linked.

williamallthing 10y ago

Great point about the need for production worthiness. FWIW the load balancing pool management, etc., in linkerd is straight from Finagle, which is probably more proven than Kubernetes, if anything (powers Twitter, Pinterest, SoundCloud, etc.) Some K8s-specific bits are fresher code, but even those have been tested at non-trivial scale, e.g. https://monzo.com/blog/2016/09/19/building-a-modern-bank-backend/ https://monzo.com/blog/2016/09/19/building-a-modern-bank-bac...

theptip 10y ago

Is the complexity of "invent[ing] its own logic to keep up to date with changing pod configurations" really that substantial? Each Service maps to an Endpoints set, which is a list of pod IP:port tuples. Seems straightforward to watch Services, and update your mesh targets when something changes. Obviously not trivial, but not a huge API surface. And the k8s API is architected to make such reimplementation as easy as possible, by composing the kube-proxy from publicly available cluster APIs. Seems like quite a thin layer of new glue to connect two moderately-well-proven components together.