Containers solve a few problems for us. They allow us to provide strong isolation guarantees for jobs and offer nice resource quota semantics (via cgroups.) The

Containers solve a few problems for us. They allow us to provide strong isolation guarantees for jobs and offer nice resource quota semantics (via cgroups.) They also serve as a good format for specifying jobs in because they can be easily passed around between nodes via docker pull and push. Our use of Docker is very similar to how you describe YARN's use of containers. We deploy on a cluster of CoreOS machines and each machine is generally as big as possible. We then pack several containers on to that same machine so that we can get as much performance as possible out of it. This also gives us an opportunity to do smart things like sharing data volumes between docker containers that are running different workloads over the same data.