3 ms·
The article is content free for people who are actually interested in resource management. I'm curious about the cluster utilization on these Google and Twitter
by vicaya 13y ago
The article is content free for people who are actually interested in resource management. I'm curious about the cluster utilization on these Google and Twitter clusters. I know Yahoo clusters had/have embarrassingly low utilization.
IFAICT, Omega and Mesos (and YARN for what it worths) cannot really handle resource overcommit effectively. I wonder what they'll call it when they reinvent a better DRS :)
- zobzu 13y agocontent free, marketing full meh :(
- jmillikin 13y agoThat depends on how you measure utilization. A naive approach is to measure idle CPU or RAM not allocated to a process. Then improving utilization metrics is simply a measure of fitting more computation onto one machine until there's not enough CPU time or RAM to fit any more. This works fine for throughput-oriented workloads, but will cause immediate conflicts when imposed on teams who value low latency (e.g. the Search team at Google). So you end up in a situation where a team is being pressured to improve utilization at the expense of metrics that they care more about, and they refuse, and that's when you get grumbling around the water cooler about "wasteful propellerheads squandering expensive hardware!" / "ignorant beancounters micromanaging things they don't understand!".