3 ms·

Mesos is a two-level scheduler, where the elected master is more of a broker for offers from agents to external schedulers that decide to act on the offer or no

Mesos is a two-level scheduler, where the elected master is more of a broker for offers from agents to external schedulers that decide to act on the offer or not. By default it's pessimistic, but this is pluggable and there's no reason you can't write an optimistic allocator module that will hand out offers to multiple schedulers.

josh2600 11y ago
Part of the beauty of Omega is that there aren't just multiple schedulers, but schedulers with different profiles (this solves the head of line problem for long-running jobs while accelerating small jobs). Mesos, from the Borg paper[0], seems to have issues with head of line problems (including going so far as to silently fail jobs without queueing them). FWIW, Omega seems like the answer if the overhead for resolving inconsistencies becomes low enough such that the benefits of optimistic scheduling outweigh the costs. [0]http://www.slideshare.net/sameertiwari33/scheduling-on-large-clusters http://www.slideshare.net/sameertiwari33/scheduling-on-large... See slide 6.
- krenoten 11y ago
  That link isn't working for me, unfortunately. I'm assuming that the HOL you're referring to stems from the assumption of the default DRF allocator of fast decisions, which can cause issues for mixed workloads if there's a lot of hoarding happening. This is something that could be addressed with an allocator module, but nobody has yet to my knowledge, leading me to think it's not a massive problem for many people. I think the people who care about this use case end up running multiple mesos clusters. One note on priorities a la Borg (what I think people are actually using at Google today, but I don't know to what extent Omega was merged with it after the Omega paper or if Borg currently runs as an Omega scheduler or what), is that it sometimes causes annoyance for people at Google, as the optimal priority is not always assigned to certain workloads, leading to starvation or overpreemption of things that were not given a healthy number. I believe that Mesos' work on oversubscription took concerns around this issue into consideration. It's really fascinating to compare the architecture of Nomad vs K8s, because both claim heavy descendence from Omega + Borg. If I squint, it seems like Nomad is more of a direct implementation based on those papers. K8s more explicitly modularizes more components, which I think is why a lot of companies are jumping behind it, as they can mold it in the ways they want.

josh2600 11y ago

Part of the beauty of Omega is that there aren't just multiple schedulers, but schedulers with different profiles (this solves the head of line problem for long-running jobs while accelerating small jobs). Mesos, from the Borg paper[0], seems to have issues with head of line problems (including going so far as to silently fail jobs without queueing them). FWIW, Omega seems like the answer if the overhead for resolving inconsistencies becomes low enough such that the benefits of optimistic scheduling outweigh the costs. [0]http://www.slideshare.net/sameertiwari33/scheduling-on-large-clusters http://www.slideshare.net/sameertiwari33/scheduling-on-large... See slide 6.

krenoten 11y ago
That link isn't working for me, unfortunately. I'm assuming that the HOL you're referring to stems from the assumption of the default DRF allocator of fast decisions, which can cause issues for mixed workloads if there's a lot of hoarding happening. This is something that could be addressed with an allocator module, but nobody has yet to my knowledge, leading me to think it's not a massive problem for many people. I think the people who care about this use case end up running multiple mesos clusters. One note on priorities a la Borg (what I think people are actually using at Google today, but I don't know to what extent Omega was merged with it after the Omega paper or if Borg currently runs as an Omega scheduler or what), is that it sometimes causes annoyance for people at Google, as the optimal priority is not always assigned to certain workloads, leading to starvation or overpreemption of things that were not given a healthy number. I believe that Mesos' work on oversubscription took concerns around this issue into consideration. It's really fascinating to compare the architecture of Nomad vs K8s, because both claim heavy descendence from Omega + Borg. If I squint, it seems like Nomad is more of a direct implementation based on those papers. K8s more explicitly modularizes more components, which I think is why a lot of companies are jumping behind it, as they can mold it in the ways they want.