4 ms·

The one thing I don't totally understand about the Kubernetes scaling issue is that you can ostensibly use Mesos as the scheduler for Kubernetes, which naively

The one thing I don't totally understand about the Kubernetes scaling issue is that you can ostensibly use Mesos as the scheduler for Kubernetes, which naively to me seems like it would allow Kubernetes to scale to the same size as Mesos, or is the DNS and Service stuff the part that doesn't scale as well?

amouat 11y ago
I think Kubernetes on top of Mesos may become a reasonably common set-up, largely because it allows diverse workloads that can drive better usage and efficiency. I'm not sure what the trade-offs are with running Kubernetes on Mesos - that would make for an interesting article.
- tknaup 11y ago
  (Mesosphere co-founder here) Running workload-specific schedulers like Kubernetes on Mesos is one of its fundamental ideas. From the paper: "It seems clear that new cluster computing frameworks will continue to emerge, and that no framework will be optimal for all applications. Therefore, organizations will want to run multiple frameworks in the same cluster, picking the best one for each application." https://people.csail.mit.edu/matei/papers/2011/nsdi_mesos.pdf https://people.csail.mit.edu/matei/papers/2011/nsdi_mesos.pd...
a1r 11y ago
Absolutely! We're actively working on this here at Mesosphere, in partnership with Google. You can check out the project at https://github.com/mesosphere/kubernetes-mesos https://github.com/mesosphere/kubernetes-mesos You might also be interested in a talk I gave at the Kubernetes launch in August about our work coming the two: https://youtu.be/aXcdHwQ5GgQ https://youtu.be/aXcdHwQ5GgQ
- TheIronYuppie 11y ago
  Full disclosure, I work at Google on Kubernetes. a1r is exactly right - Mesos does great in virtualizing your data center, Kubernetes is a framework on top of that.
theptip 11y ago
There are a bunch of scale limits in Kubernetes itself that are being chipped away at; the bottleneck is not the scheduler, so plugging a new scheduler in won't make the system more performant. Here's a blog discussing the current state of K8s scale: http://blog.kubernetes.io/2015/09/kubernetes-performance-measurements-and.html http://blog.kubernetes.io/2015/09/kubernetes-performance-mea... Note that Bob Wise at Samsung has been driving some horizontal scale testing, and has got a 1000-node cluster up and running, so that's a current "best case" scale number.
- robszumski 11y ago
  Here's Bob's video of a 1000 machines running on CoreOS' Tectonic product: https://www.youtube.com/watch?v=1bHUsBhPL20 https://www.youtube.com/watch?v=1bHUsBhPL20 (work at CoreOS)

amouat 11y ago

I think Kubernetes on top of Mesos may become a reasonably common set-up, largely because it allows diverse workloads that can drive better usage and efficiency. I'm not sure what the trade-offs are with running Kubernetes on Mesos - that would make for an interesting article.

tknaup 11y ago
(Mesosphere co-founder here) Running workload-specific schedulers like Kubernetes on Mesos is one of its fundamental ideas. From the paper: "It seems clear that new cluster computing frameworks will continue to emerge, and that no framework will be optimal for all applications. Therefore, organizations will want to run multiple frameworks in the same cluster, picking the best one for each application." https://people.csail.mit.edu/matei/papers/2011/nsdi_mesos.pdf https://people.csail.mit.edu/matei/papers/2011/nsdi_mesos.pd...

a1r 11y ago

Absolutely! We're actively working on this here at Mesosphere, in partnership with Google. You can check out the project at https://github.com/mesosphere/kubernetes-mesos https://github.com/mesosphere/kubernetes-mesos You might also be interested in a talk I gave at the Kubernetes launch in August about our work coming the two: https://youtu.be/aXcdHwQ5GgQ https://youtu.be/aXcdHwQ5GgQ

TheIronYuppie 11y ago
Full disclosure, I work at Google on Kubernetes. a1r is exactly right - Mesos does great in virtualizing your data center, Kubernetes is a framework on top of that.

theptip 11y ago

There are a bunch of scale limits in Kubernetes itself that are being chipped away at; the bottleneck is not the scheduler, so plugging a new scheduler in won't make the system more performant. Here's a blog discussing the current state of K8s scale: http://blog.kubernetes.io/2015/09/kubernetes-performance-measurements-and.html http://blog.kubernetes.io/2015/09/kubernetes-performance-mea... Note that Bob Wise at Samsung has been driving some horizontal scale testing, and has got a 1000-node cluster up and running, so that's a current "best case" scale number.

robszumski 11y ago
Here's Bob's video of a 1000 machines running on CoreOS' Tectonic product: https://www.youtube.com/watch?v=1bHUsBhPL20 https://www.youtube.com/watch?v=1bHUsBhPL20 (work at CoreOS)