4 ms·

OT: is this really "simple" to setup or do you mean that once it's running that it's simple? or what do you mean with simplicity. I actually looking forward to

by merb 9y ago

OT: is this really "simple" to setup or do you mean that once it's running that it's simple? or what do you mean with simplicity. I actually looking forward to use k8s in our network, too. But I think the simplest solution for HA k8s is using keepalived paired with nginx/haproxy on the master level and add a keepalived-vip for every service. (at least for our 3 node master, 3 node worker setup) (I evaluated the calico + bgp option, too but it looked way to complex, as a k8s starter)

mrmondo 9y ago
Yes, it was very simple to setup, deploy and understand on an ongoing process, BGP is incredibly simple by design. I’m heading to bed now (AEDST), but check out: - https://docs.projectcalico.org/latest/getting-started/kubernetes/tutorials/simple-policy https://docs.projectcalico.org/latest/getting-started/kubern... - https://kubernetes.io/docs/tasks/administer-cluster/calico-network-policy/ https://kubernetes.io/docs/tasks/administer-cluster/calico-n... And there’s plenty of other resources out there. IMO if you’re designing a highly available platform, BGP is one of the simplest parts of that complex system. In my experience you’re likely to run into much more finicky problems with higher layer systems including failover delays, reliance on casting systems and state drift.
scurvy 9y ago
Kubernetes is very complicated, but BGP is not. If the curve for BGP is too steep, you can do OSPF inside your racks with BGP announcements to spine. This is pretty common as OSPF can use area 0 with little to zero configuration. BGP leaf to spine is standard is layer 3 networks.
murali-reddy 9y ago
As an alternative approach may I suggest to look at the way kube-router does. It basically leverages BGP and ECMP to provide HA. https://cloudnativelabs.github.io/post/2017-11-01-kube-high-available-ingress/ https://cloudnativelabs.github.io/post/2017-11-01-kube-high-...

by merb 9y ago

mrmondo 9y ago

Yes, it was very simple to setup, deploy and understand on an ongoing process, BGP is incredibly simple by design. I’m heading to bed now (AEDST), but check out: - https://docs.projectcalico.org/latest/getting-started/kubernetes/tutorials/simple-policy https://docs.projectcalico.org/latest/getting-started/kubern... - https://kubernetes.io/docs/tasks/administer-cluster/calico-network-policy/ https://kubernetes.io/docs/tasks/administer-cluster/calico-n... And there’s plenty of other resources out there. IMO if you’re designing a highly available platform, BGP is one of the simplest parts of that complex system. In my experience you’re likely to run into much more finicky problems with higher layer systems including failover delays, reliance on casting systems and state drift.

scurvy 9y ago

Kubernetes is very complicated, but BGP is not. If the curve for BGP is too steep, you can do OSPF inside your racks with BGP announcements to spine. This is pretty common as OSPF can use area 0 with little to zero configuration. BGP leaf to spine is standard is layer 3 networks.

murali-reddy 9y ago

As an alternative approach may I suggest to look at the way kube-router does. It basically leverages BGP and ECMP to provide HA. https://cloudnativelabs.github.io/post/2017-11-01-kube-high-available-ingress/ https://cloudnativelabs.github.io/post/2017-11-01-kube-high-...