5 ms·

Any load balancer can be configured or modified to target routable Pod IP addresses and skip node ports altogether. You'll have to integrate with the Kubernetes

Any load balancer can be configured or modified to target routable Pod IP addresses and skip node ports altogether. You'll have to integrate with the Kubernetes Endpoints API[1] and support dynamic backends. Another option would be to leverage Kubernetes' DNS and the SRV records[2] backing each service. The reason node ports are used in the Cloud today is because most Cloud load balancing solutions only target VMs, not arbitrary endpoints such as containers, a limitation that will go away over time. [1] Envoy with Kubernetes Endpoints integration: https://github.com/kelseyhightower/kubernetes-envoy-sds https://github.com/kelseyhightower/kubernetes-envoy-sds [2] https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/ https://kubernetes.io/docs/concepts/services-networking/dns-...

codesuki 9y ago
Sorry to turn this into a different direction but I wanted to ask you about packaging + deployment. I am about to try using helm for packaging my Kubernetes configs to make use of its templating. Being able to include Kubernetes changes in the release makes it less common to forget some new env variable etc. The only thing I don't like is that it replaces kubectl in a way. And some comments here speak of problems during deploy and rollback which makes me wary. What are some best practices around that topic? I was also thinking of deploying with spinnaker but could not confirm it works with helm. Any info is appreciated! P.S. have you thought about making a redis-cluster example repo? Redis 4.0 has a new `cluster_announce_ip` setting with which I made it work, but I still don't like my setup 100%.
- deleted 9y ago
  [deleted]
- bacongobbler 9y ago
  This is a question you should ask in #helm-users in kubernetes.slack.com. Lots of Helm users are more than happy to help answer questions like this. :)
sandGorgon 9y ago
Hi Kelsey, thanks for replying. You are definitely k8s secret weapon ;) I know about this. However, ultimately the question is that Kubernetes is not a gradual scale up solution for most people. I have to be prepared to deal with building my own load balancer. Basically, I cannot do an on-metal deployment very easily. Most of the questions on k8s slack for metal deployments were - how do I set this up with few tweaks like ssl pass through and source ip preservation. It is not easy. Either you build your own load balancer or you use a cloud provided one. Now, Ingresses are not pleasant. I'm not sure about the state of source ip preservation, but last I remember that the nginx ingress had still not surfaced ssl_preread_server_name to the ingress configuration. Now, what would have been nice is if it was ingress-all-the-way-down : ingress with something like istio/linkerd, maybe it is possible. Tl;Dr - I'm not github. I can't build my own load balancer. Give me something that works out of the box. Yes, I know it may go down - I'll survive. Docker Swarm does this.
- hardymansen 9y ago
  For nginx. load balancing to pods from outside kubernetes is one solution that i think is quite elegant on premise. https://github.com/unibet/ext_nginx https://github.com/unibet/ext_nginx
- shaklee3 9y ago
  If you use a statefulset, there is no load balancing, regardless of bare metal or cloud. Every pod has a DNS record you can use to address all other pods, and it's carried over in case of a pod failure. Are you looking for a load balancer or not? If you are, as the other person mentioned, you can use the nginx one.
shaklee3 9y ago
Since Kelsey won't pitch his book, I'll do it for him. Coming out soon: https://www.amazon.com/Kubernetes-Running-Dive-Future-Infrastructure/dp/1491935677 https://www.amazon.com/Kubernetes-Running-Dive-Future-Infras...

codesuki 9y ago

Sorry to turn this into a different direction but I wanted to ask you about packaging + deployment. I am about to try using helm for packaging my Kubernetes configs to make use of its templating. Being able to include Kubernetes changes in the release makes it less common to forget some new env variable etc. The only thing I don't like is that it replaces kubectl in a way. And some comments here speak of problems during deploy and rollback which makes me wary. What are some best practices around that topic? I was also thinking of deploying with spinnaker but could not confirm it works with helm. Any info is appreciated! P.S. have you thought about making a redis-cluster example repo? Redis 4.0 has a new `cluster_announce_ip` setting with which I made it work, but I still don't like my setup 100%.

deleted 9y ago
[deleted]
bacongobbler 9y ago
This is a question you should ask in #helm-users in kubernetes.slack.com. Lots of Helm users are more than happy to help answer questions like this. :)

sandGorgon 9y ago

Hi Kelsey, thanks for replying. You are definitely k8s secret weapon ;) I know about this. However, ultimately the question is that Kubernetes is not a gradual scale up solution for most people. I have to be prepared to deal with building my own load balancer. Basically, I cannot do an on-metal deployment very easily. Most of the questions on k8s slack for metal deployments were - how do I set this up with few tweaks like ssl pass through and source ip preservation. It is not easy. Either you build your own load balancer or you use a cloud provided one. Now, Ingresses are not pleasant. I'm not sure about the state of source ip preservation, but last I remember that the nginx ingress had still not surfaced ssl_preread_server_name to the ingress configuration. Now, what would have been nice is if it was ingress-all-the-way-down : ingress with something like istio/linkerd, maybe it is possible. Tl;Dr - I'm not github. I can't build my own load balancer. Give me something that works out of the box. Yes, I know it may go down - I'll survive. Docker Swarm does this.

hardymansen 9y ago
For nginx. load balancing to pods from outside kubernetes is one solution that i think is quite elegant on premise. https://github.com/unibet/ext_nginx https://github.com/unibet/ext_nginx
shaklee3 9y ago
If you use a statefulset, there is no load balancing, regardless of bare metal or cloud. Every pod has a DNS record you can use to address all other pods, and it's carried over in case of a pod failure. Are you looking for a load balancer or not? If you are, as the other person mentioned, you can use the nginx one.

shaklee3 9y ago

Since Kelsey won't pitch his book, I'll do it for him. Coming out soon: https://www.amazon.com/Kubernetes-Running-Dive-Future-Infrastructure/dp/1491935677 https://www.amazon.com/Kubernetes-Running-Dive-Future-Infras...