3 ms·
> But there is no equivalent system for whole clusters. To transition from one cluster to another, you have to handle everything that k8s gives you without rely
by edude03 3y ago
> But there is no equivalent system for whole clusters. To transition from one cluster to another, you have to handle everything that k8s gives you without relying on k8s for it
Ah, but there are. Maybe not a cluster orchestrator that would let you hot swap a single cluster, but if you're only running one cluster it's inherently a pet already.
One common way that I've personally used to solve this problem is use Federated Clusters (kubefed) to run multiple clusters with global load balancing so that even when one entire cluster goes away its traffic is just shifted to the other cluster (whether because of upgrades or unexpected outages of the control plane)
- simiones 3y agoGlobal load balancing is not enough, is it? If a request to save a new piece of info is served by one cluster, it still needs to update all the other clusters so that they can call retrieve it. So now you also need global data replication. And the data storage service shards need a name that's easily resolvable, so you need global name resolution across your clusters. And you need something to monitor the workloads so it can trigger failover, so you need global health checks too. And when you do an upgrade, you need to coordinate cordoning off the clusters etc - so you need global state as well. So overall, you need all, or most, of the features of kubernetes in your cluster federation solution. So why then not make each of these individual kubernetes clusters really small, say a single-node cluster each, and rely on the global infra to manage between them? Hey-presto, your "cattle farm" is now a single big pet.