5 ms·
Docker Part One: Containers
- wmf 12y agoI don't think this is a very good article in general and especially not for HN where Docker has been covered in detail already. (The history of the LXC vs. libcontainer split is pretty far inside baseball and not even that interesting IMO.) Speaking of Docker, I did find some interesting repos today: https://github.com/socketplane/socketplane https://github.com/socketplane/socketplane and https://github.com/ibuildthecloud/only-docker https://github.com/ibuildthecloud/only-docker
- ldlework 12y agoThese are definitely exciting times for container networking!
- SwellJoe 12y agoI gasped at this assertion: "Virtuozzo, by the way, is still the best container system available. Unfortunately, it isn't open source, and that means you have to pay if you want to use it. Virtuozzo, by the way, has the distinction of being the only halfway decent container solution to (currently) run natively on Windows." I don't even know how to process a comment like that about Virtuozzo. The fragility of Virtuozzo and OpenVZ is infamous. We get more support queries about our software running on OpenVZ and Virtuozzo containers than all other virtualization types combined (our stuff runs on a few hundred thousand hosting servers, we have a really good sample size). For years, whenever we get a mysterious "this thing stops for no reason" or "this error doesn't make any sense" or "sometimes it works and sometimes it doesn't" support request, literally our first clarifying question is, "Is this an OpenVZ or Virtuozzo system?" It is, frankly, broken by design, with regard to memory allocation.
- saraid216 12y ago> I don't even know how to process a comment like that about Virtuozzo. Well, it'd help if you offered a counterexample to the original claim refuting it.
- SwellJoe 12y agoI would recommend anything other than Virtuozzo/OpenVZ, because of broken memory allocation. Virtuozzo allows hosts to oversell RAM, without properly handling over-allocation. So, memory allocations fail while reporting available memory, making it difficult to troubleshoot, and difficult for users to predict what their VM can accommodate. If intermittent failures without useful error messages is the worst kind of failure (and it is, IMHO), then Virtuozzo intentionally subjects its users to the worst kind of failure. Unless you know how to spot this kind of problem, it's easy to bang your head on it for days or weeks; and we've had users who were ecstatic when we diagnose their problem and provide the knowledge they need to spot it in the future (though resolving it may be more tricky than diagnosing it). Even without the host intentionally overselling, we've seen this behavior happen, because allocation seems to require contiguous blocks of memory (or something else that means that even with enough available memory allocations can fail). I am not deeply familiar with the code, only with spotting this problem in the wild. I've seen assertions, spanning years (including from Parallels), that it's been fixed, or that as long as the host isn't overselling RAM it won't happen. But, we still get these support requests regularly. I'm not an expert on their software, but I'm an expert on spotting when it's their software that is causing our customer's despair. (Note that the support requests are usually about Postfix, Apache, MySQL or other services "crashing"; it rarely manifests with our software specifically, since our software rarely needs to allocate large amounts of memory.) If you happen to be stuck on a Virtuozzo or OpenVZ system, and have mysterious failures of services, you can find out of it is this problem by checking /proc/bc/* or /proc/user_beancounters.
- snom320 12y agoI don't really get what your problem is? You can disable the memory limitations if your use case requires it, or set the limits as high as you want. How else would you suggest that OpenVZ should handle over-allocation? It is, after all, one of the points of virtualization (and the reason why you can get such containers for cheap). You can't really blame OpenVZ if the users or sysadmins are unaware that the container has hard limits and that you're able to see if you've run into them using /proc/user_beancounters. Of course, if OpenVZ fails to allocate memory when you're under the limit, that's a separate problem that should be adressed (although I've yet to run into that one).
- dmpk2k 12y agoI agree so much with the above. Been there, done that, run an illumos distro now; it's like night and day. Calling Virtuozzo the best container system is a bad joke. Same applies to OpenVZ.
- patrickg_zill 12y agoI don't know when they changed from "user_beancounters" to "VSwap" but VSwap fixes all these problems. It is also a problem with FreeSwitch (the VOIP server software, like Asterisk). Under VSwap-enabled OpenVZ there is not 1 problem.
- SwellJoe 12y agoI wish they would tell their customers. We have not stopped seeing these types of problem reports. And, I've heard that the problem was solved numerous times in the past, so I'm hesitant to believe that it is finally fixed now.
- xyby 12y agoI have read a lot about Docker now. Here and elsewhere. I even have been to Docker talks in my town. Still I do not get what the usecase is. Everytime I ask that, I get the answer "When you want to run applications seperated from each other in their own environment". Yeah, I get that. But what is the real world use case for that? Who wants to do that? Now I will get answers like "Everybody who wants to run applications in containers" :) Yeah ok. But who? Do bloggers need Docker? People who run onlineshops? Would Ycombinator benefit from running Hacker News inside a Docker container?
- cleverjake 12y agosysadmins who have to run services. the same way you separate you model from your view from your controller, you can separate your various services from one another.
- drinchev 12y agoImagine you have 20 services that need to run with your website ( 5 database daemons, 5 nodejs apps, 5 ngnix instances, 2 dns servers, 2 mail smtpd daemons, 1 git repository ). Your infrastructure consists of 5+ virtual servers that host your services. Each virtual server might be database only or db + dns or web + dns or ... you got the point. What docker allows you to do is to isolate all those 20 services in separate containers that if any of them is being hacked your attacker will end up compromising only that service without any access to the main server. What docker allows you to do is to version control the configuration and the container for the different services in a way that if you need to reinstall a service ( redeploy your app ) it is less of a hassle and since it is containeraized you will not be compromising your other services while doing that. etc.
- bildung 12y agoMaybe I misunderstood GP's question, but what you describe has been done with countless other solutions for years, why should I specifically use docker and not stay with kvm, or bsd jails, or...?
- 12y ago
- markdown 12y agoI reached part 3 and still don't know a thing about Docker other than it's history and how its the best thing since sliced bread and it's amazing that it is what it is because its amazing. Does part 4 finally deliver on the promise in the title?
- deleted 12y ago[deleted]
- WestCoastJustin 12y agoBit of a personal plug, but I've put together a screencast about Docker, and people seem to think it is helpful at giving an overview @ https://sysadmincasts.com/episodes/31-introduction-to-docker https://sysadmincasts.com/episodes/31-introduction-to-docker
- bwohlergo 12y agono, it does not.
- crooksey 12y agoI'm sorry, if you cannot see the benefits to running programs inside a container, then you have been reading the wrong article. I have been using FreeBSD jails for years, and now on Linux deployments, docker as well. Lets say you have a server, it runs a mail server, database server, web server (proxy) and application server. If they all run without containers, if one service gets compromised, and a root exploit is found, that's it game over. If you have a service that starts eating up memory, with proper configuration, it can't overload the root server. Basically each jail/container can only see itself and any exploits cannot effect the host system, or other jails/containers (when configured correctly). It also allows for easy expansion, when one jailed/containered service gets to large for the server, just move it to another server easily and quickly. It also allows for speedy deployment, with docker you can bundle everything on your laptop, create an image, then ship this straight to an external host like EC2/Goole Cloud (for example), with the addition of pre built containers for Django/Rails/Postgres/MySQL etc, it creates a ready working environment for developers who might not be to hot on configuring systems. The "shipping" ability to docker is a by product of the container which is another added benefit. There are loads more features than what I have stated here, this is just a very brief summary.
- nhaehnle 12y agoIf they all run without containers, if one service gets compromised, and a root exploit is found, that's it game over. To be fair, if a kernel-level root exploit is found, it's probably also game over for containers. It's possible to have root exploits that cannot escape containers due to UID virtualization or whatever, but typically(?) root exploits are based on being able to mess with kernel memory, in which case escaping a container should also be possible.
- mrweasel 12y ago> if one service gets compromised, and a root exploit is found, that's it game over. For root exploits isn't Docker toast as well? I haven't followed Docker in much details, but does Docker actually promise that commands run as root will be contained?
- 12y ago
- polysics 12y agoI have a more general question for you: I am a senior developer who is starting to figure out devops because I need to deploy two different products we built. I have limited Chef and Ansible experience and pretty good Unix skills. Should I start out with Docker or am I missing some crucial pieces I should go through first? The use case is "media processing services" in one case, and a sports betting game in the other.
- cgb_ 12y agoDocker has a lot of rough edges and it's most useful in large scale-out deployments. In-band devops tools like Chef and Ansible are relatively agnostic to the hypervisor or container solution, so if you only need a few OS instances & managed configuration, I suggest sticking with Chef/Ansible/Puppet (I prefer ansible :)). You can always scale out with docker later and I know at least ansible are adressing this use case (http://www.ansible.com/docker http://www.ansible.com/docker).
- 23david 12y agoMy advice is to stick with a mature configuration management solution like Chef/Puppet/Saltstack/Ansible, and then add on Docker as an optimization. Saltstack in particular has really good Docker integration at this point, which may be useful. YMMV.
- vacri 12y agoI used puppet (masterless) in a small company, and in my new company I'm using Ansible and Docker. Puppet and Chef are great, but they're overkill unless you have large infrastructure. If there's only one of you doing 'ops', you don't have large enough infra (in my opinion). Ansible is easy to configure and learn, and doesn't wrap you in dependency hell like puppet can. I would avoid Docker until you have a use-case that Docker solves for you. Use your preferred distro's packaging technique. It can be a bit of a pain (.deb in particular) to start with, but once you're familiar, it's portable and manageable with your OS's native tools. Things 'just work', and don't need an agent to be installed and kept up-to-date. If something goes wrong, it's in your OS's standard logs, not some strange arcane location. And most importantly, you have a bigger pool of interneteers to help learn from and troubleshoot with. I can't complain about the Docker IRC channel, good experience there, but there's still fewer of them. If you are a .debber and want your own repo, check out Aptly, which makes life much easier than previous repo software on offer. http://www.aptly.info/ http://www.aptly.info/ TL;DR: Docker is 'another thing to learn', and it's not trivial. Put it off until it solves a problem for you; there's plenty of other stuff to get up to speed with.
- mardiros 12y agoCan someone could explain why "If you use Puppet, chances are you love Docker ..." (from Part 2)?
- cgb_ 12y agoOne of the things I've found interesting about rapidly popular opensource solutions in the last 1-2 years is how quickly venture cap funding comes in and drives the direction of future development. Docker has a whole ecosystem of bolt-on github projects that fill-gaps that the Docker project hasn't had time themselves to fill because they are too busy organising conferences, withholding features until the keynote of said conference, and working out how to spend the millions of dollars on future development. Even the gap-filler github projects are getting VC funding (see http://weaveblog.com/2014/12/03/announcing-weaveworks-and-our-series-a-funding/ http://weaveblog.com/2014/12/03/announcing-weaveworks-and-ou...). Weave will either become the standard networking stack, or be made irrelevant by an in-tree featureful implementation. So Docker are on this tightrope trying to innovate whilst avoiding stepping on the toes of their VC-funded 'partners', so their response was simple-core with a plugin framework for partners to integrate into. Anyone else picturing fragmentation in 6-12 months time?