3 ms·

Would an overlay network with a shared secret for encryption of pod-to-pod networking be another solution to this problem? I feel like the ideal should involve

Would an overlay network with a shared secret for encryption of pod-to-pod networking be another solution to this problem? I feel like the ideal should involve keeping the key material in as few places as possible.

theptip 10y ago
Yes, that's an option -- the simplest option would be a single overlay network, with the node-node tunnels encrypted using IPSec or similar (https://github.com/coreos/flannel/issues/6 https://github.com/coreos/flannel/issues/6 or https://www.weave.works/documentation/net-latest-how-it-works/net-latest-encryption/ https://www.weave.works/documentation/net-latest-how-it-work... or https://github.com/projectcalico/felix/issues/997 https://github.com/projectcalico/felix/issues/997). I think this would be tricky to configure in GKE though. With a secret-per-pod, your key material lives in in the etcd on the API server, and gets mounted in a tmpfs on each pod that is given the secret. Only Pods in the Secret's namespace can access the Secret, so if you have RBAC configured correctly it should be possible to lock this down tightly to only the code that needs the Secret. (I'm not sure how to do this in GKE; I'm currently treating each cluster as a single security domain).
- A-F1V3 10y ago
  If you are open to using an (awesome) vendor for this kind of thing, I would highly recommend taking a look at Backplane (https://www.backplane.io/ https://www.backplane.io/). It's as simple as running a sidecar Backplane agent container along side any container that you want to route http traffic to and then shape the traffic via their API (it becomes your load balancer as well). They automatically provision Lets Encrypt certificates for your endpoints as well, so you don't have to worry about any of that. We have been using it at Mux in production for months and have been very happy with the results. Backplane also has some other nice built in features like blue/green deploys and built in OAuth support that are really nice to have out of the box.

theptip 10y ago

Yes, that's an option -- the simplest option would be a single overlay network, with the node-node tunnels encrypted using IPSec or similar (https://github.com/coreos/flannel/issues/6 https://github.com/coreos/flannel/issues/6 or https://www.weave.works/documentation/net-latest-how-it-works/net-latest-encryption/ https://www.weave.works/documentation/net-latest-how-it-work... or https://github.com/projectcalico/felix/issues/997 https://github.com/projectcalico/felix/issues/997). I think this would be tricky to configure in GKE though. With a secret-per-pod, your key material lives in in the etcd on the API server, and gets mounted in a tmpfs on each pod that is given the secret. Only Pods in the Secret's namespace can access the Secret, so if you have RBAC configured correctly it should be possible to lock this down tightly to only the code that needs the Secret. (I'm not sure how to do this in GKE; I'm currently treating each cluster as a single security domain).

A-F1V3 10y ago
If you are open to using an (awesome) vendor for this kind of thing, I would highly recommend taking a look at Backplane (https://www.backplane.io/ https://www.backplane.io/). It's as simple as running a sidecar Backplane agent container along side any container that you want to route http traffic to and then shape the traffic via their API (it becomes your load balancer as well). They automatically provision Lets Encrypt certificates for your endpoints as well, so you don't have to worry about any of that. We have been using it at Mux in production for months and have been very happy with the results. Backplane also has some other nice built in features like blue/green deploys and built in OAuth support that are really nice to have out of the box.