5 ms·
The truth is gRPC, like kubernetes, was built with decades of lessons of RPC framework inside a container oriented distributed environment; and more importantly
by justicezyx 7y ago
The truth is gRPC, like kubernetes, was built with decades of lessons of RPC framework inside a container oriented distributed environment; and more importantly, gRPC is the blessed framework inside Google as well, meaning it's qualified to power the largest and most complex distributed systems in the world (I think it'd safe to omit 'one of' here), which in comparison is not the case for kubernetes.
Addition: Borg and kubernetes are designed with similar goals but different emphasis. They are like complementing twins had different personalities. For this I recommend Min Cai's Kubecon'18 presentation about peleton [1], the slide is titled "comparison of cluster manager architecture".
[1] https://kccna18.sched.com/event/GrTx/peloton-a-unified-scheduler-for-web-scale-workloads-on-mesos-kubernetes-min-cai-nitin-bahadur-uber https://kccna18.sched.com/event/GrTx/peloton-a-unified-sched...
- shereadsthenews 7y agoWait, I don’t get it. Kubernetes:Borg::gRPC:Stubby. Google uses gRPC internally to the same extent that they use Kubernetes internally, i.e. hardly at all.
- mehrdada 7y agoThis analogy is very misleading. Kubernetes is probably never going to run any real workload internally at Google, but gRPC powers all external APIs of Google Cloud, and increasing other Google properties (e.g. ads, assistant), used by mobile apps like Duo and Allo, and have some big internal service use cases. The reason Stubby still dominates internally is simply taking lots of time to migrate to gRPC that might be hard to justify, but I do see gRPC being used very widely internally at Google; it’s simply a matter of time. I don’t see that happening to Kubernetes; it’s a joke when compared to Borg. Google aside, many other companies like Dropbox rely on gRPC extensively to successfully run infrastructure: https://static.sched.com/hosted_files/grpconf19/f7/Courier%20—%20gRPC%20Conf%202019.pdf https://static.sched.com/hosted_files/grpconf19/f7/Courier%2...
- enitihas 7y agoCuriously, what do you find lacking in Kubernetes compared to Borg?
- mehrdada 7y agoScale, for one thing. Kubernetes is not a bad system but it’s not designed to run Google.
- int0x80 7y agoCan you expand on that, please?
- q3k 7y agohttps://kubernetes.io/docs/setup/cluster-large/ https://kubernetes.io/docs/setup/cluster-large/ More specifically, "No more than 5000 nodes" and "No more than 150000 total pods" is fairly limiting to large (Google-large) clusters.
- int0x80 7y agothanks for the answer.
- dilyevsky 7y agoI used to work on one of Borg teams at Google and now run Kubernetes platform. Nearly every Kubernetes component (node, master, networking) will melt down at fraction of Borg scale. It's not even close.
- int0x80 7y agothanks for the answer.
- Thaxll 7y agoWhat google system runs more than 5k tasks on a single cluster?
- techslave 7y agogrpc was built way before containers were a thing. jails and zones were barely out of the gate at the time.
- justicezyx 7y agoYou are taking about stubby, I would guess. Borg is cirra 2003, pb/stubby was before that. Gfs probably was similar in timing as stubby. And many other cluster level foundations. In the end, Borg is the true corner stone that ties everything together and completes the Google infrastructure puzzle (or the modern global scale cluster computing).