5 ms·

At their most basic level, Kubernetes [1] and Mesos [2] both use a client/server type architecture, where you install a client on many compute nodes, then the s

At their most basic level, Kubernetes [1] and Mesos [2] both use a client/server type architecture, where you install a client on many compute nodes, then the server part farms out jobs (in the form of containers) to these client compute nodes. Mesos does not do scheduling, and is pretty low-level in terms of interfaces, so you would typically run some type of software which talks to it via API, like Marathon [3], Aurora [4], or Hadoop [5]. Then Marathon/Aurora/Hadoop/etc tells Mesos to farm out compute jobs to these end client nodes (aka schedules). Complexity can quickly go up depending on your hosting environment, scale, and HA requirements. There is actually a really good high-level overview diagram of what a mesos/hadoop/zookeeper setup looks like here [6]. The stack looks something like this for Kubernetes/Mesos (top down): - Kubernetes and Mesos (client/server packages depending on node type) - Docker (container engine) - OS (Ubuntu/RHEL/etc) What are some use-cases? - you have more work than can fit into one server - need to distribute load across N+ nodes - Google heavily uses containers (not k8s, but that inspired these patterns) - Gmail/Search/etc all run in containers [7] - Apple, Twitter, and Airbnb are running Mesos today [8, 9] There are a bunch of revolving services, like: - distributed key/values stores (etcd/zookeeper) - load balancers - image registries - user interfaces - cli tools - logging/monitoring/alerting - etc But, to answer your question, the main difference between Kubernetes and Mesos, is that Kubernetes offers an opinionated workflow, built-in scheduler, and patterns for how containers are deployed into this cluster of compute notes. The pattern is baked in from the start via Pods, Labels, Services, and Replication Controllers. It also helps to know that Kubernetes comes from Google, where they have been running containers in-house, so much of this workflow (pods, services, replication controllers, etc), comes from their internal use-cases. That's the 10,000 foot view. [1] https://github.com/GoogleCloudPlatform/kubernetes https://github.com/GoogleCloudPlatform/kubernetes [2] http://mesos.apache.org/ http://mesos.apache.org/ [3] https://mesosphere.github.io/marathon/ https://mesosphere.github.io/marathon/ [4] http://aurora.apache.org/ http://aurora.apache.org/ [5] https://hadoop.apache.org/ https://hadoop.apache.org/ [6] http://mesos.apache.org/assets/img/documentation/architecture3.jpg http://mesos.apache.org/assets/img/documentation/architectur... [7] http://www.wired.com/2013/03/google-borg-twitter-mesos/ http://www.wired.com/2013/03/google-borg-twitter-mesos/ [8] http://www.infoq.com/news/2015/05/mesos-powers-apple-siri http://www.infoq.com/news/2015/05/mesos-powers-apple-siri [9] https://www.youtube.com/watch?v=E4lxX6epM_U https://www.youtube.com/watch?v=E4lxX6epM_U

DannoHung 11y ago
One small correction: Mesos is a scheduler. It doesn't natively ship with any end-user framework to access the scheduling though (you are supposed to write your own framework which uses the Mesos API). Marathon is a generic end-user framework for accessing those functions and runs on top of Mesos. I think it's also interesting to note that Mesos can be used as the Kubernetes minion scheduler backend. And for very large collections of compute nodes, this is reputedly a good choice (though I don't have any personal experience to back that assessment up). https://github.com/mesosphere/kubernetes-mesos https://github.com/mesosphere/kubernetes-mesos
- WestCoastJustin 11y ago
  Oh, thanks. Will check that out.
- presspot 11y ago
  In the parlance of Mesos, the Mesos kernel is an "allocator" (in that it assembles and allocates all of the compute resources) and the frameworks are the "schedulers" in that they request and consume "resource offers" (made by the allocator) and then schedule tasks to be run on those accepted resources.
- SEJeff 11y ago
  Actually that isn't entirely true. mesos-execute will run any shell command and it shows up as a framework in the framework api/ui tab on the mesos master.
kiyoto 11y ago
>- logging/monitoring/alerting As a maintainer of Fluentd [1], an open source log collector now integrated with Kubernetes [2] and Docker [3], happy to see this out =) [1] https://www.fluentd.org https://www.fluentd.org [2] http://blog.raintown.org/2014/11/logging-kubernetes-pods-using-fluentd.html http://blog.raintown.org/2014/11/logging-kubernetes-pods-usi... [3] http://blog.treasuredata.com/blog/2015/07/07/collecting-docker-logs-with-fluentd/ http://blog.treasuredata.com/blog/2015/07/07/collecting-dock...
rsync 11y ago

DannoHung 11y ago

One small correction: Mesos is a scheduler. It doesn't natively ship with any end-user framework to access the scheduling though (you are supposed to write your own framework which uses the Mesos API). Marathon is a generic end-user framework for accessing those functions and runs on top of Mesos. I think it's also interesting to note that Mesos can be used as the Kubernetes minion scheduler backend. And for very large collections of compute nodes, this is reputedly a good choice (though I don't have any personal experience to back that assessment up). https://github.com/mesosphere/kubernetes-mesos https://github.com/mesosphere/kubernetes-mesos

WestCoastJustin 11y ago
Oh, thanks. Will check that out.
presspot 11y ago
In the parlance of Mesos, the Mesos kernel is an "allocator" (in that it assembles and allocates all of the compute resources) and the frameworks are the "schedulers" in that they request and consume "resource offers" (made by the allocator) and then schedule tasks to be run on those accepted resources.
SEJeff 11y ago
Actually that isn't entirely true. mesos-execute will run any shell command and it shows up as a framework in the framework api/ui tab on the mesos master.

kiyoto 11y ago

>- logging/monitoring/alerting As a maintainer of Fluentd [1], an open source log collector now integrated with Kubernetes [2] and Docker [3], happy to see this out =) [1] https://www.fluentd.org https://www.fluentd.org [2] http://blog.raintown.org/2014/11/logging-kubernetes-pods-using-fluentd.html http://blog.raintown.org/2014/11/logging-kubernetes-pods-usi... [3] http://blog.treasuredata.com/blog/2015/07/07/collecting-docker-logs-with-fluentd/ http://blog.treasuredata.com/blog/2015/07/07/collecting-dock...

rsync 11y ago