4 ms·

Kubernetes is not really a container orchestration platform. It helps organize containers into logical groups with dependencies, but orchestrating resources on

Kubernetes is not really a container orchestration platform. It helps organize containers into logical groups with dependencies, but orchestrating resources on your physical machines is well within Mesos' domain. Keep in mind that Kubernetes and Mesos aren't mutually exclusive. Kubernetes has been [implemented](https://github.com/mesosphere/kubernetes-mesos https://github.com/mesosphere/kubernetes-mesos) on Mesos. My opinion is that Kubernetes is closer in identity to Docker: tons of OSS excitement, many companies jumping on board, and really pluggable. As to whether its going to become the critical infrastructure piece that gets enterprise to pay $$$... that's a stretch.

josephjacks 12y ago
Abhay, perhaps you can disclose in your biased answer that you are employed by Mesosphere?
- notacoward 12y ago
  Perhaps you should have disclosed in your answer that you're a founder of a Kubernetes-focused startup.
  - josephjacks 12y ago
    Noted! Apologies.
    - notacoward 12y ago
      Thanks, and cheers. :)
      - josephjacks 12y ago
        Cheers. :-)
jsmthrowaway 12y ago
> Kubernetes is not really a container orchestration platform. Since Kubernetes schedules, I'm curious what makes you say this. Kubernetes without Mesos works perfectly well for the use cases that both CoreOS and Google envision.
- abhay_agarwal 12y ago
  To what scale? A lot of tools out there help you schedule containers, but to how many machines, containers, and level of resource utilization? Perhaps I meant that it's _technically_ an orchestration platform, but it's not at the level of quality that is required for large-scale orchestrations (i.e. Borg, Mesos). And yes, as noted above I work for Mesosphere. YMMV.
  - chuhnk 12y ago
    I think everyone is clear on the fact that after 9 months Kubernetes is not yet as mature as the 4-5 year old Mesos. But people love the project and the direction that it's headed in, to such a degree that there are partnerships forming, startups emerging and many many people contributing to the project. Many of us who will adopt Kubernetes are not looking to scale workloads across a Twitter or Google scale environment. We might have a few hundred machines or less, some might go beyond that but in all honesty I'd say for the time being people can resort to 100 machine clusters and manage multiple clusters. Don't get me wrong, I really like mesos. I think it's a fantastic piece of technology and I really want to see it do extremely well. I just think Kubernetes is something a little more accessible to the wider developer community. Plus we get to see it grow from infancy and help contribute to that growth. AND its written in Go, which is a major plus.
    - krenoten 12y ago
      It's a plus until it isn't :P
  - jsmthrowaway 12y ago
    Maybe it's just me, but perhaps I like the company who actually wrote the scheduler powering the service you mentioned, has run it in production for over a decade and gathered petabytes of knowledge about how to refine it, has been working for several years on its replacement, and has woken up in the last couple years and realized that while their infrastructure has been their competitive advantage to this point there is value in the whole industry having access to to their technology. Maybe I like their expertise and I'm not ready to condemn Kubernetes and its quality just yet because it's young and who knows where they'll go. Maybe they'll separate Omega from google3 and land it in Kubernetes. Then what? Mesos and Mesosphere talk a lot about scale, with the implication that the technology you're modeling powers the entire Google fleet in one big harmonious system. It doesn't. There's a fairly hard upper limit on the system you mentioned in terms of scale, which is one of the motivations for Omega. The system you mentioned also fits into an ecosystem from a DLM[0] to a RPC system to a build system and on up. Pieces of that are gradually being open-sourced, too, like Bazel. Google is clearly becoming aware that their infrastructure being generally available will benefit the industry enormously, and regardless of my personal feelings on their business model and exploitation of user data, that is a good thing. If you're telling me that Mesos can do better than all of that expertise, better than John Wilkes, better than all of that, to the point where you don't even consider Kubernetes on the same path as Mesos, I don't know what to tell you other than come back to reality. This seems to be a rut with Apache projects that model Google systems in some way: Zookeeper, Mesos, Hadoop, Flume. All of them look like if I described the systems in question over a very blurry fax, and the open-source community did its best. That's not outright criticism, it's just having the unique experience of having seen their Google counterparts; it's a good thing they exist, but there are absolutely opportunities to do better, Mesos included. I get what you're trying to do with Mesos, but I strongly disagree with your characterization of Mesosphere as a datacenter operating system. It's disingenuous, and positioning Mesosphere as the only company who can do hyperscale in containers is a tough sell. Last time I used Mesos, in 2012, it could barely handle 100 machines and I had to write shell scripts to even get a simple static application scheduled. I then threw 10,000 jobs at it and the scheduler hung and took down the entire cluster. Clearly you have come a long way since then, but it's apparently easy for you to forget that given the age of Kubernetes. [0]: You'll pardon me if the thought of Zookeeper doesn't excite me as the backbone of the entire system.

josephjacks 12y ago

Abhay, perhaps you can disclose in your biased answer that you are employed by Mesosphere?

notacoward 12y ago
Perhaps you should have disclosed in your answer that you're a founder of a Kubernetes-focused startup.
- josephjacks 12y ago
  Noted! Apologies.
  - notacoward 12y ago
    Thanks, and cheers. :)
    - josephjacks 12y ago
      Cheers. :-)

jsmthrowaway 12y ago

> Kubernetes is not really a container orchestration platform. Since Kubernetes schedules, I'm curious what makes you say this. Kubernetes without Mesos works perfectly well for the use cases that both CoreOS and Google envision.

abhay_agarwal 12y ago
To what scale? A lot of tools out there help you schedule containers, but to how many machines, containers, and level of resource utilization? Perhaps I meant that it's _technically_ an orchestration platform, but it's not at the level of quality that is required for large-scale orchestrations (i.e. Borg, Mesos). And yes, as noted above I work for Mesosphere. YMMV.
- chuhnk 12y ago
  I think everyone is clear on the fact that after 9 months Kubernetes is not yet as mature as the 4-5 year old Mesos. But people love the project and the direction that it's headed in, to such a degree that there are partnerships forming, startups emerging and many many people contributing to the project. Many of us who will adopt Kubernetes are not looking to scale workloads across a Twitter or Google scale environment. We might have a few hundred machines or less, some might go beyond that but in all honesty I'd say for the time being people can resort to 100 machine clusters and manage multiple clusters. Don't get me wrong, I really like mesos. I think it's a fantastic piece of technology and I really want to see it do extremely well. I just think Kubernetes is something a little more accessible to the wider developer community. Plus we get to see it grow from infancy and help contribute to that growth. AND its written in Go, which is a major plus.
  - krenoten 12y ago
    It's a plus until it isn't :P
- jsmthrowaway 12y ago
  Maybe it's just me, but perhaps I like the company who actually wrote the scheduler powering the service you mentioned, has run it in production for over a decade and gathered petabytes of knowledge about how to refine it, has been working for several years on its replacement, and has woken up in the last couple years and realized that while their infrastructure has been their competitive advantage to this point there is value in the whole industry having access to to their technology. Maybe I like their expertise and I'm not ready to condemn Kubernetes and its quality just yet because it's young and who knows where they'll go. Maybe they'll separate Omega from google3 and land it in Kubernetes. Then what? Mesos and Mesosphere talk a lot about scale, with the implication that the technology you're modeling powers the entire Google fleet in one big harmonious system. It doesn't. There's a fairly hard upper limit on the system you mentioned in terms of scale, which is one of the motivations for Omega. The system you mentioned also fits into an ecosystem from a DLM[0] to a RPC system to a build system and on up. Pieces of that are gradually being open-sourced, too, like Bazel. Google is clearly becoming aware that their infrastructure being generally available will benefit the industry enormously, and regardless of my personal feelings on their business model and exploitation of user data, that is a good thing. If you're telling me that Mesos can do better than all of that expertise, better than John Wilkes, better than all of that, to the point where you don't even consider Kubernetes on the same path as Mesos, I don't know what to tell you other than come back to reality. This seems to be a rut with Apache projects that model Google systems in some way: Zookeeper, Mesos, Hadoop, Flume. All of them look like if I described the systems in question over a very blurry fax, and the open-source community did its best. That's not outright criticism, it's just having the unique experience of having seen their Google counterparts; it's a good thing they exist, but there are absolutely opportunities to do better, Mesos included. I get what you're trying to do with Mesos, but I strongly disagree with your characterization of Mesosphere as a datacenter operating system. It's disingenuous, and positioning Mesosphere as the only company who can do hyperscale in containers is a tough sell. Last time I used Mesos, in 2012, it could barely handle 100 machines and I had to write shell scripts to even get a simple static application scheduled. I then threw 10,000 jobs at it and the scheduler hung and took down the entire cluster. Clearly you have come a long way since then, but it's apparently easy for you to forget that given the age of Kubernetes. [0]: You'll pardon me if the thought of Zookeeper doesn't excite me as the backbone of the entire system.