4 ms·
>Kubernetes hasn’t been able to scale to the large clusters that Uber requires, i.e. 10,000-plus, nor does it support elastic resource sharing. It is also not t
by elvinyung 8y ago
>Kubernetes hasn’t been able to scale to the large clusters that Uber requires, i.e. 10,000-plus, nor does it support elastic resource sharing. It is also not the ideal scheduler for batch workloads due to the high-churn nature of batch jobs.
Feels like a semi-weird justification for NIH. Isn't there pretty active work in addressing most/all of those objections in k8s?
- tapoxi 8y agoDidn't they also have a weird "Postgres can't suit our needs so we're switching to MySQL" argument?
- cphoover 8y agoyurp
- cdoxsey 8y agoYes, though the problems are real. In its current form Kubernetes can only handle a cluster size of 5,000 nodes: https://kubernetes.io/docs/setup/cluster-large/ https://kubernetes.io/docs/setup/cluster-large/ and high-churn batch workloads don't work well. Kubernetes does support elastic resource sharing (assuming I understand what they need) via horizontal pod scaling and the cluster autoscaler: https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler https://github.com/kubernetes/autoscaler/tree/master/cluster.... Although there are rough edges here, I was surprised by the quality of the code in the cluster autoscaler. I kept thinking of new edge cases and expected it to fall over, only to find out that they'd thought of that problem and it handled it gracefully. For batch workloads something like Argo can help: https://github.com/argoproj/argo https://github.com/argoproj/argo. So basically I agree, it probably would've made more sense to build something on top of Kubernetes. But Uber's a big company with a lot of engineers so I guess they have the resources to roll their own solution.