just a quick tip: check out weave for managing networking, IMO it's much easier and I'm not even talking about the docker plugin. just use weave without the pl

just a quick tip: check out weave for managing networking, IMO it's much easier and I'm not even talking about the docker plugin. just use weave without the plugin and it works amazingly well. for the configuration aspect, just use dockerfiles and a system like mesos and marathon for deploying production. you can then write simple restful API commands to launch different environments with marathon. build all your docker containers to act as single server instances, i.e., spark on one container, SQL on another. tie in a default install of cadvisor on all mesos clients and you can have easy monitoring as well.