5 ms·

Kubernetes is written in go

by eggie5 10y ago

kasey_junk 10y ago
Is kubernetes widely used at google?
- vitus 10y ago
  In short, yes. How else would we provide scalable containerized services? Docker?? [0] [0] Don't be silly, of course we rolled our own.
- lobster_johnson 10y ago
  Not widely, as far as I know. Google has two existing systems, Borg and Omega, whose ideas were distilled into Kubernetes. According to an answer on Quora in May [1], Kubernetes is only used for some things that run on the Google Cloud Platform. Some more background here [2]. [1] https://www.quora.com/Does-Google-use-the-Open-Source-Kubernetes-or-a-version-of-Borg-for-their-own-container-management https://www.quora.com/Does-Google-use-the-Open-Source-Kubern... [2] http://queue.acm.org/detail.cfm?id=2898444 http://queue.acm.org/detail.cfm?id=2898444
  - Cyph0n 10y ago
    That's pretty disappointing honestly. So which container management platform is actually used at scale?
    - lobster_johnson 10y ago
      Open source ones? Or did you mean internally within Google? Kubernetes is growing in popularity. Among older systems, Mesos (with Marathon, usually) seems to be quite popular.
      - Cyph0n 10y ago
        No, in general. Is there a solution that is actually well tested in production? I can go through the marketing spiel claiming that the solution in question is "scalable", but I'd rather see that in practice.
        lobster_johnson 10y ago
        Kubernetes is definitely being used at scale in production. Spotify uses Helios [1] to run their stuff. We evaluated it briefly and decided it was too limited and immature compared to Kubernetes. Mesos + a framework such as Marathon or Aurora was the most needy choice before Kubernetes came on the scene. Mesos probably scales farther than Kubernetes in terms of pure cluster sizes, but it also depends on what framework you use on top (Mesos itself is just a scheduler). I don't know if any of them are as flexible as Kubernetes in terms of things like volume management, config/secret management and security. It's also worth pointing out that Kubernetes can run on Mesos. [1] https://github.com/spotify/helios https://github.com/spotify/helios
        bogomipz 10y ago
        >"Mesos + a framework such as Marathon or Aurora was the most needy choice before Kubernetes came on the scene." How is Mesos "needy"? Can you elaborate? Needy has a negative connotation and it's not word I would necessarily associate with the Apache Mesos project. I've run a number of clusters in production now for just over a year with Marathon and it pretty much "just works." I have done 5 or 6 rolling upgrades now without issues. I haven't found it to be needy at all, quite the opposite, its been rock solid and the management overhead has been nominal. >" I don't know if any of them are as flexible as Kubernetes in terms of things like volume management, config/secret management and security." I think that Mesos is actually more flexible as it allows you to cherry pick the non-scheduler specific components to fit your use case. As an example for secrets management you can use something like Consul Vault or integrate Keywhiz or completely roll your own. I feel like with Kubenetes you buy into the "whole thing". Using the example of secret management - you have one choice for secret management and the last time I checked they were stored in etcd in clear text. So if that doesn't fit your security requirements it seemed like you were out of luck. Mesos also has a nice for for persistent volumes: See: http://mesos.apache.org/documentation/latest/persistent-volume/ http://mesos.apache.org/documentation/latest/persistent-volu... and: http://schd.ws/hosted_files/mesosconeu2016/08/MesosConEurope2016PPVv1.0.pdf http://schd.ws/hosted_files/mesosconeu2016/08/MesosConEurope...

Kubernetes is written in go

by eggie5 10y ago

Kubernetes is written in go

kasey_junk 10y ago

Is kubernetes widely used at google?

vitus 10y ago
In short, yes. How else would we provide scalable containerized services? Docker?? [0] [0] Don't be silly, of course we rolled our own.
lobster_johnson 10y ago
Not widely, as far as I know. Google has two existing systems, Borg and Omega, whose ideas were distilled into Kubernetes. According to an answer on Quora in May [1], Kubernetes is only used for some things that run on the Google Cloud Platform. Some more background here [2]. [1] https://www.quora.com/Does-Google-use-the-Open-Source-Kubernetes-or-a-version-of-Borg-for-their-own-container-management https://www.quora.com/Does-Google-use-the-Open-Source-Kubern... [2] http://queue.acm.org/detail.cfm?id=2898444 http://queue.acm.org/detail.cfm?id=2898444
- Cyph0n 10y ago
  That's pretty disappointing honestly. So which container management platform is actually used at scale?
  - lobster_johnson 10y ago
    Open source ones? Or did you mean internally within Google? Kubernetes is growing in popularity. Among older systems, Mesos (with Marathon, usually) seems to be quite popular.
    - Cyph0n 10y ago
      No, in general. Is there a solution that is actually well tested in production? I can go through the marketing spiel claiming that the solution in question is "scalable", but I'd rather see that in practice.
      - lobster_johnson 10y ago
        Kubernetes is definitely being used at scale in production. Spotify uses Helios [1] to run their stuff. We evaluated it briefly and decided it was too limited and immature compared to Kubernetes. Mesos + a framework such as Marathon or Aurora was the most needy choice before Kubernetes came on the scene. Mesos probably scales farther than Kubernetes in terms of pure cluster sizes, but it also depends on what framework you use on top (Mesos itself is just a scheduler). I don't know if any of them are as flexible as Kubernetes in terms of things like volume management, config/secret management and security. It's also worth pointing out that Kubernetes can run on Mesos. [1] https://github.com/spotify/helios https://github.com/spotify/helios
        bogomipz 10y ago
        >"Mesos + a framework such as Marathon or Aurora was the most needy choice before Kubernetes came on the scene." How is Mesos "needy"? Can you elaborate? Needy has a negative connotation and it's not word I would necessarily associate with the Apache Mesos project. I've run a number of clusters in production now for just over a year with Marathon and it pretty much "just works." I have done 5 or 6 rolling upgrades now without issues. I haven't found it to be needy at all, quite the opposite, its been rock solid and the management overhead has been nominal. >" I don't know if any of them are as flexible as Kubernetes in terms of things like volume management, config/secret management and security." I think that Mesos is actually more flexible as it allows you to cherry pick the non-scheduler specific components to fit your use case. As an example for secrets management you can use something like Consul Vault or integrate Keywhiz or completely roll your own. I feel like with Kubenetes you buy into the "whole thing". Using the example of secret management - you have one choice for secret management and the last time I checked they were stored in etcd in clear text. So if that doesn't fit your security requirements it seemed like you were out of luck. Mesos also has a nice for for persistent volumes: See: http://mesos.apache.org/documentation/latest/persistent-volume/ http://mesos.apache.org/documentation/latest/persistent-volu... and: http://schd.ws/hosted_files/mesosconeu2016/08/MesosConEurope2016PPVv1.0.pdf http://schd.ws/hosted_files/mesosconeu2016/08/MesosConEurope...