4 ms·

Ok, I'll bite. I posted this in another thread, but check this out: https://stackoverflow.com/questions/50195896/how-do-i-get-one-pod-to-network-to-another-pod

Ok, I'll bite. I posted this in another thread, but check this out: https://stackoverflow.com/questions/50195896/how-do-i-get-one-pod-to-network-to-another-pod-in-kubernetes-simple https://stackoverflow.com/questions/50195896/how-do-i-get-on.... That's the amount of crap I waded through trying to rubber-ducky myself into figuring out how to get two pods to talk to each other. In the end, I copied a solution my friend had gotten, and it's still not great. I'd love to be able to use Ingress or Calico or Fabric or something to get path routing to work in Kubernetes, but unfortunately all the examples I've seen online suffer from too much specificity. Which is the Kubernetes problem - it can do everything so trying to get it to do the one thing you want is hard. Here is the kubernetes cluster I ended up building. If anyone has any ideas on how to add path routing let me know - https://github.com/patientplatypus/KubernetesMultiPodCommunication https://github.com/patientplatypus/KubernetesMultiPodCommuni...

jchw 8y ago
I think part of the problem is that I can't immediately understand what is actually being done. You say you want a per se React frontend to talk to a Node.js backend. But that's not really a pod-to-pod communication issue; both frontend and backend will be communicating with the user's browser, outside the cluster. Secondly, you deployed an Nginx ingress controller. You don't need to deploy more than one of these in your whole cluster, so you can go ahead and separate this from your program's deployment manifests. Typically, cluster add-ons are installed by running kubectl -f with a URL to a GitHub raw URL, or, if you want to be much cleaner, using Helm (basically, a package manager. It installs with one command and then you can use it to install things into your Kubernetes cluster easily, such as an Nginx ingress controller.) If you're wondering why the process is such a mess, it's probably just because Ingress is still new. In the future, support for more environments will probably come by default, without needing to install third party controllers. Already, in Google Cloud, GKE clusters come ready with an Ingress Controller that creates a Google Cloud load balancer. As a side note, I found that the nginx ingress controller was not working by default in my cluster. I noticed some errors in the logs and had to change a parameter with Helm. Don't recall what it was, unfortunately.
- anoncoward1234 8y ago
  The problem with adding the Ingress controller via Helm (and with a lot of other Kubernetes abstractions) is that it spits out a lot of code that is then difficult or impossible to reason about. `Helm Ingress --whateversyntaxdefualt` spits out 1000+ lines of Ingress controller code that is essentially two deployments with a health check and auto spin up, but it's complicated. In production can I use this or is there a security hole in there? What if the ports the health check are using overlap with other ports I have assigned somewhere else? What if something equally silly? Maybe Kubernetes is new so that's why it's so wild west, but it really feels like a pile of bandaids right now.
  - merb 8y ago
    You should not use Ingress. Use Nginx or Haproxy and do it on K8S like you would do it normally and you can scale your nginx haproxy with kubectl scale --replicas=2 deploy nginx On the outside use metallb which than gets you a single IP which is highly available either via L2 or with bgp (if you have bgp gear) if you are not on the cloud. What people do wrong with k8s is that they think different, which is silly. k8s just exposes a "managed vm" where you can built stuff like you would do on vmware vApps.
    - jchw 8y ago
      I disagree with two statements: > You should not use Ingress Why not? It allows you to route your applications automagically with Kubernetes objects. Instead of writing nginx configurations that do what you want, you can just describe how you want your routing to work. I don't see why that isn't useful. > k8s just exposes a "managed vm" where you can built stuff like you would do on vmware vApps. Pods aren't even containers, less VMs. They're namespaces with containers in them. Secondly, while you can use those pods like VM and boot systemd or whatever in them, that's not really the way you're intended to use Docker. Just to quote an official source: https://docs.docker.com/config/containers/multi-service_container/ https://docs.docker.com/config/containers/multi-service_cont... > It is generally recommended that you separate areas of concern by using one service per container. Instead of treating Kubernetes like a VM manager, the actual intended way to use it is to treat it like a task manager, like systemd or what have you. The pods are meant to represent individual services, and containers individual processes. The problem Kubernetes solves is managing applications, not machines. The difference is not merely semantic rambling; it's a paradigm shift.
dilatedmind 8y ago
within a cluster, service names are addressable, eg curl http://backend/api/whatever http://backend/api/whatever I only glanced at your repo, but it sounds like you have an ingress problem. That is, you don't really have two pods communicating, but you need to hit your backend service from outside the cluster. Under the hood this is always accomplished with a node port on a service- a single port on any node in your cluster will forward to said service. k8s has integration with cloud providers like aws to hook all this up for you, but all its doing is setting up an elb which load balances to that port on every node in your cluster.
gfddfh 8y ago
Ok, so I'm on mobile, so I don't have a chance to look through your cluster atm, but this looks easy enough. Can you tell me how you set up kubernetes (minikube, Google container engine, kops, etc)?
justinsaccount 8y ago
> the amount of crap I waded through trying to rubber-ducky myself into figuring out how to get two pods to talk to each other well, it's more the amount of crap you waded through trying to figure out that you were not actually trying to get two pods to talk to each other at all. Path routing should work, if it's not, what you should do is exec into the nginx pod and inspect the nginx config that the nginx ingress controller generated. Traefik has an example of this that is basically what you are doing: https://github.com/containous/traefik/blob/master/examples/k8s/cheeses-ingress.yaml https://github.com/containous/traefik/blob/master/examples/k... https://github.com/containous/traefik/blob/master/examples/k8s/cheese-services.yaml https://github.com/containous/traefik/blob/master/examples/k... The main thing I see you doing wrong is you are using 'type: LoadBalancer' for everything when that is exactly what you don't want.

jchw 8y ago

I think part of the problem is that I can't immediately understand what is actually being done. You say you want a per se React frontend to talk to a Node.js backend. But that's not really a pod-to-pod communication issue; both frontend and backend will be communicating with the user's browser, outside the cluster. Secondly, you deployed an Nginx ingress controller. You don't need to deploy more than one of these in your whole cluster, so you can go ahead and separate this from your program's deployment manifests. Typically, cluster add-ons are installed by running kubectl -f with a URL to a GitHub raw URL, or, if you want to be much cleaner, using Helm (basically, a package manager. It installs with one command and then you can use it to install things into your Kubernetes cluster easily, such as an Nginx ingress controller.) If you're wondering why the process is such a mess, it's probably just because Ingress is still new. In the future, support for more environments will probably come by default, without needing to install third party controllers. Already, in Google Cloud, GKE clusters come ready with an Ingress Controller that creates a Google Cloud load balancer. As a side note, I found that the nginx ingress controller was not working by default in my cluster. I noticed some errors in the logs and had to change a parameter with Helm. Don't recall what it was, unfortunately.

anoncoward1234 8y ago
The problem with adding the Ingress controller via Helm (and with a lot of other Kubernetes abstractions) is that it spits out a lot of code that is then difficult or impossible to reason about. `Helm Ingress --whateversyntaxdefualt` spits out 1000+ lines of Ingress controller code that is essentially two deployments with a health check and auto spin up, but it's complicated. In production can I use this or is there a security hole in there? What if the ports the health check are using overlap with other ports I have assigned somewhere else? What if something equally silly? Maybe Kubernetes is new so that's why it's so wild west, but it really feels like a pile of bandaids right now.
- merb 8y ago
  You should not use Ingress. Use Nginx or Haproxy and do it on K8S like you would do it normally and you can scale your nginx haproxy with kubectl scale --replicas=2 deploy nginx On the outside use metallb which than gets you a single IP which is highly available either via L2 or with bgp (if you have bgp gear) if you are not on the cloud. What people do wrong with k8s is that they think different, which is silly. k8s just exposes a "managed vm" where you can built stuff like you would do on vmware vApps.
  - jchw 8y ago
    I disagree with two statements: > You should not use Ingress Why not? It allows you to route your applications automagically with Kubernetes objects. Instead of writing nginx configurations that do what you want, you can just describe how you want your routing to work. I don't see why that isn't useful. > k8s just exposes a "managed vm" where you can built stuff like you would do on vmware vApps. Pods aren't even containers, less VMs. They're namespaces with containers in them. Secondly, while you can use those pods like VM and boot systemd or whatever in them, that's not really the way you're intended to use Docker. Just to quote an official source: https://docs.docker.com/config/containers/multi-service_container/ https://docs.docker.com/config/containers/multi-service_cont... > It is generally recommended that you separate areas of concern by using one service per container. Instead of treating Kubernetes like a VM manager, the actual intended way to use it is to treat it like a task manager, like systemd or what have you. The pods are meant to represent individual services, and containers individual processes. The problem Kubernetes solves is managing applications, not machines. The difference is not merely semantic rambling; it's a paradigm shift.

dilatedmind 8y ago

within a cluster, service names are addressable, eg curl http://backend/api/whatever http://backend/api/whatever I only glanced at your repo, but it sounds like you have an ingress problem. That is, you don't really have two pods communicating, but you need to hit your backend service from outside the cluster. Under the hood this is always accomplished with a node port on a service- a single port on any node in your cluster will forward to said service. k8s has integration with cloud providers like aws to hook all this up for you, but all its doing is setting up an elb which load balances to that port on every node in your cluster.

gfddfh 8y ago

Ok, so I'm on mobile, so I don't have a chance to look through your cluster atm, but this looks easy enough. Can you tell me how you set up kubernetes (minikube, Google container engine, kops, etc)?

justinsaccount 8y ago

> the amount of crap I waded through trying to rubber-ducky myself into figuring out how to get two pods to talk to each other well, it's more the amount of crap you waded through trying to figure out that you were not actually trying to get two pods to talk to each other at all. Path routing should work, if it's not, what you should do is exec into the nginx pod and inspect the nginx config that the nginx ingress controller generated. Traefik has an example of this that is basically what you are doing: https://github.com/containous/traefik/blob/master/examples/k8s/cheeses-ingress.yaml https://github.com/containous/traefik/blob/master/examples/k... https://github.com/containous/traefik/blob/master/examples/k8s/cheese-services.yaml https://github.com/containous/traefik/blob/master/examples/k... The main thing I see you doing wrong is you are using 'type: LoadBalancer' for everything when that is exactly what you don't want.