4 ms·

The funny thing about Kubernetes is that I read somewhere that it's not actually widely used within Google (please correct me if I'm wrong). Given that's it's

The funny thing about Kubernetes is that I read somewhere that it's not actually widely used within Google (please correct me if I'm wrong). Given that's it's a complicated piece of software and that Google has extremely complicated requirements, that's pretty concerning. To me, we've barely starting to solve deployments, the idea of Kubernetes seems a little ahead of it's time. If you're using GPC I do hear it's really great. But then, what's the huge benefit of Kubernetes in the first place? A tiny bit less of vendor lock in?

bhouston 8y ago
Kubernetes is newer than Google's Borg infrastructure I understand. https://kubernetes.io/blog/2015/04/borg-predecessor-to-kubernetes/ https://kubernetes.io/blog/2015/04/borg-predecessor-to-kuber...
mooreds 8y ago
>I read somewhere that it's not actually widely used within Google Borg and Container Engine: https://www.quora.com/Does-Google-use-the-Open-Source-Kubernetes-or-a-version-of-Borg-for-their-own-container-management https://www.quora.com/Does-Google-use-the-Open-Source-Kubern...
kozikow 8y ago
If someone thinks kubernetes is complicated, they would cry during their first month with the borg.
- ebikelaw 8y ago
  I would argue that borg has much less initial complexity than kubernetes. Because of Google's integrated systems you can take any local binary and run it in borg with a five-line config file and one command. You can't get started that quickly with kubernetes. If you understand the implications of running N replicas of a command on <= N machines in the cloud, borg does exactly what you want. Most of the people I met at Google complaining about borg just didn't understand how a program gets executed at all ... borg was the least of their problems. Also borg is easier to pronounce.
  - kozikow 8y ago
    You can use "kubectl run" in k8s for one line run. Now if it runs it runs. Go figure why your borg job is getting preempted.
    - ebikelaw 8y ago
      You can use kubectl run to make something run, only after you've got all your initial authentication/authorization squared away, and your nodes are deployed. Assuming you work at Google you can "borgcfg up" your job and it will run under your role, make authenticated RPCs under your own authority, using your personal freebie quota which is infinite. You can get preempted; that's the price of infinite free resources. Anybody who complains about that should be fired.
tbrock 8y ago
This is the ultimate key feature of Kubernetes. It’s 1000x better than a hosted closed source solution because you can run it entirely locally. It’s amazing that the hosted option 100% mirrors the self hosted option which mirrors dev.
atombender 8y ago
Google uses Borg internally, and Kubernetes is really their third container orchestration system. After Borg came Omega, which was never deployed, but ended up being a test bed for a lot of innovations that were folded back into Borg. But Borg is a decade old and has a lot of warts (according to its designers), and with Kubernetes they aimed to learn from their mistakes and improve on the design. As far as I can tell, the aim with Kubernetes was never to replace Borg at Google -- Google is far too invested in Borg, and it would take a considerable engineering effort to migrate away from it. Rather, developers at Google saw an opportunity to create an open source version based on what they had learned and help the world along in adopting the same engineering principles as Google has long practiced. Not all altruistic notions, of course -- Google benefits from the commoditization of containers indirectly, by undermining competitors such as AWS (where containers are still not well-supported) and making their own cloud the best fit for Kubernetes. Google does run stuff on Kubernetes, via GKE. As I understand it, new products are encouraged to run on GCP. I don't know how many applications they run, however. Maybe someone from Google can comment.
- erikb 8y ago
  The devs totally forgot that in a normal environment you don't have the 10.000 other internal Google tools, though.

bhouston 8y ago

Kubernetes is newer than Google's Borg infrastructure I understand. https://kubernetes.io/blog/2015/04/borg-predecessor-to-kubernetes/ https://kubernetes.io/blog/2015/04/borg-predecessor-to-kuber...

mooreds 8y ago

>I read somewhere that it's not actually widely used within Google Borg and Container Engine: https://www.quora.com/Does-Google-use-the-Open-Source-Kubernetes-or-a-version-of-Borg-for-their-own-container-management https://www.quora.com/Does-Google-use-the-Open-Source-Kubern...

kozikow 8y ago

If someone thinks kubernetes is complicated, they would cry during their first month with the borg.

ebikelaw 8y ago
I would argue that borg has much less initial complexity than kubernetes. Because of Google's integrated systems you can take any local binary and run it in borg with a five-line config file and one command. You can't get started that quickly with kubernetes. If you understand the implications of running N replicas of a command on <= N machines in the cloud, borg does exactly what you want. Most of the people I met at Google complaining about borg just didn't understand how a program gets executed at all ... borg was the least of their problems. Also borg is easier to pronounce.
- kozikow 8y ago
  You can use "kubectl run" in k8s for one line run. Now if it runs it runs. Go figure why your borg job is getting preempted.
  - ebikelaw 8y ago
    You can use kubectl run to make something run, only after you've got all your initial authentication/authorization squared away, and your nodes are deployed. Assuming you work at Google you can "borgcfg up" your job and it will run under your role, make authenticated RPCs under your own authority, using your personal freebie quota which is infinite. You can get preempted; that's the price of infinite free resources. Anybody who complains about that should be fired.

tbrock 8y ago

This is the ultimate key feature of Kubernetes. It’s 1000x better than a hosted closed source solution because you can run it entirely locally. It’s amazing that the hosted option 100% mirrors the self hosted option which mirrors dev.

atombender 8y ago

Google uses Borg internally, and Kubernetes is really their third container orchestration system. After Borg came Omega, which was never deployed, but ended up being a test bed for a lot of innovations that were folded back into Borg. But Borg is a decade old and has a lot of warts (according to its designers), and with Kubernetes they aimed to learn from their mistakes and improve on the design. As far as I can tell, the aim with Kubernetes was never to replace Borg at Google -- Google is far too invested in Borg, and it would take a considerable engineering effort to migrate away from it. Rather, developers at Google saw an opportunity to create an open source version based on what they had learned and help the world along in adopting the same engineering principles as Google has long practiced. Not all altruistic notions, of course -- Google benefits from the commoditization of containers indirectly, by undermining competitors such as AWS (where containers are still not well-supported) and making their own cloud the best fit for Kubernetes. Google does run stuff on Kubernetes, via GKE. As I understand it, new products are encouraged to run on GCP. I don't know how many applications they run, however. Maybe someone from Google can comment.

erikb 8y ago
The devs totally forgot that in a normal environment you don't have the 10.000 other internal Google tools, though.