4 ms·

Yeah with bare mesos I don't have to do any of that, I just accept resource offers and release them when I'm done. The scheduler decides how much it will scale,

Yeah with bare mesos I don't have to do any of that, I just accept resource offers and release them when I'm done. The scheduler decides how much it will scale, not mesos. We like to create a new framework for each instance of the job (so we can track slow runs, trial-run new versions), so we can have anywhere between 0 and 50 similar frameworks all running and vying for 100% of the cluster. Naturally, we have cluster-level scaling, so we add nodes until we hit our max spend per hour, and then jobs take a bit longer to complete.

hardwaresofton 8y ago
It was kind of hard to understand what you were describing without reading the Mesos Architecture documentation[0], but I think I get it now. When you say "you" just accept resource offers, you mean the frameworks (in mesos terminology) you're deploying correct? Weirdly enough, Mesos resembles a system I was building in my head that I thought could compete with Kubernetes... Taking the resource supplier (agents in mesos-speak) and consumer (frameworks in mesos-speak) paradigm to the extreme. [0]: https://mesos.apache.org/documentation/latest/architecture/ https://mesos.apache.org/documentation/latest/architecture/
- falsedan 8y ago
  Essentially, yeah. You don't really deploy a framework, you just start its scheduler. I recommend the technical paper, it's clear and short[0]. 0: https://people.eecs.berkeley.edu/~alig/papers/mesos.pdf https://people.eecs.berkeley.edu/~alig/papers/mesos.pdf
- halbritt 8y ago
  I've been running a reasonably large kubernetes cluster (~600 cores) in production for the last 6 months or so. I've had to deal with hotspots, both IO, CPU and load (which could be either). Getting requests and limits dialed has been a bit of a challenge. I worked with Mesos a bit last year and miss the sophistication of the scheduler. Granted, I never ran it at the same scale, but I felt like the approach was much easier to grok. I've seen a few projects floating around meant help dynamically tune resource utilization in k8s. By this time next year, I imagine those will be fairly commonplace. I'm definitely looking forward to using descheduler: https://github.com/kubernetes-incubator/descheduler https://github.com/kubernetes-incubator/descheduler

hardwaresofton 8y ago

It was kind of hard to understand what you were describing without reading the Mesos Architecture documentation[0], but I think I get it now. When you say "you" just accept resource offers, you mean the frameworks (in mesos terminology) you're deploying correct? Weirdly enough, Mesos resembles a system I was building in my head that I thought could compete with Kubernetes... Taking the resource supplier (agents in mesos-speak) and consumer (frameworks in mesos-speak) paradigm to the extreme. [0]: https://mesos.apache.org/documentation/latest/architecture/ https://mesos.apache.org/documentation/latest/architecture/

falsedan 8y ago
Essentially, yeah. You don't really deploy a framework, you just start its scheduler. I recommend the technical paper, it's clear and short[0]. 0: https://people.eecs.berkeley.edu/~alig/papers/mesos.pdf https://people.eecs.berkeley.edu/~alig/papers/mesos.pdf
halbritt 8y ago
I've been running a reasonably large kubernetes cluster (~600 cores) in production for the last 6 months or so. I've had to deal with hotspots, both IO, CPU and load (which could be either). Getting requests and limits dialed has been a bit of a challenge. I worked with Mesos a bit last year and miss the sophistication of the scheduler. Granted, I never ran it at the same scale, but I felt like the approach was much easier to grok. I've seen a few projects floating around meant help dynamically tune resource utilization in k8s. By this time next year, I imagine those will be fairly commonplace. I'm definitely looking forward to using descheduler: https://github.com/kubernetes-incubator/descheduler https://github.com/kubernetes-incubator/descheduler