7 ms·
In practice, containers are process groups with access restrictions. In theory, though, containers are simply an optimization of virtual machines, and are best
by derefr 13y ago
In practice, containers are process groups with access restrictions.
In theory, though, containers are simply an optimization of virtual machines, and are best understood that way. If you can use virtual machines to solve a problem--and your virtual machines are all generally based on a recent Linux--then you can achieve the same thing, using far fewer resources, by using containers.
- 3am 13y agoI think the virtual machine analogy is a bit dicey because lxc+cgroups/Docker containers have to be much more aware of the host kernel api and devices. edit: yes derefr, I agree. My point was also if there was a distribution or kernel specific Ruby or Ruby library bug, for example, then you might have to account for it in your code if you ran in containers (if you don't control the host on which your code is running), whereas you control the whole stack in the virtual machine.
- derefr 13y agoBut it's not idiomatic to take advantage of those close ties (by, say, having two containers share a named pipe mounted from the host.) I suppose what you mean is that containers aren't applicable in every instance that VMs are--like, say, providing VPS instances, where some of the instances might want to do funny fiddling things with their virtualized "hardware." This is true. But for the situations where the applicability of VMs and containers intersect--using them to run Heroku-like isolated "app slugs", for example--then you can think of containers as lighter-weight VMs, and you won't really go wrong.
- k3oni 13y agoYep, this is more like it
- gaius 13y agoBut there should be no such thing as a VPS. You should be able to just give n people a shell on a box and quotas, permissions etc would suffice. That you can't is a shortfall of the OS and a hypervisor is a bandaid.
- tbrownaw 13y agoWhich could be made to work until someone wants to run an alternative OS (ie, not Linux) or use features that are only available in newer kernel versions. Or until someone realizes that a shared kernel with a security policy is a larger attack surface than a tiny hypervisor with no concept of sharing.
- dpark 13y agoAnd what's your security model? Each user that needs to run services has to run them under their user account? Or are you going to also allocate some number of service accounts to each user and pay the cost to manage these accounts and the combined quotas they entail? Now how do your users' service accounts share resources? Someone will need a web server running under one account and a job processor running under a different account (different accounts in order to allow for limited privileges). Are they restricted to communicating through named pipes (hope no one else binds the pipe)? What if they want to share some disk-based resources? Shove them all in a group and set someone's home directory to 770? How do you handle it when one of your users needs a SQL server? Do they need to install this in their home directory, too? And lock down their install so that hopefully other users on the box can't call it? What about port binding? Just luck of the draw, and everyone binds on what they want? Then they file a request for you to forward to the port they happened to grab? And they hope that they never lose the port due to a server reboot, or a service crash? People could work in the environment you describe, but it would be miserable. And you (the server owner) would have to sink a massive amount of resources into maintenance. This isn't a shortcoming of the OS. It's a fundamental mismatch between the goal of sharing a server among many users and giving those users sufficient control to build what they need.
- chongli 13y agoHow do you handle it when one of your users needs a SQL server? Do they need to install this in their home directory, too? And lock down their install so that hopefully other users on the box can't call it? NixOS can handle situations like this. It enables per-user package management with deterministic, immutable installs. If multiple users install the same package, it only puts one copy on the system. Each user gets their own environment where they can customize the package without affecting the other users.
- MrBuddyCasino 13y agoI think we've used something similar some time ago (maybe XEN?) and had the following issues, how does Docker compare: - what happens when one container changes the system clock? - are iptables rules per container or global?
- ambiate 13y agoAs far as I understand, the clock and such are tied to the base host. Typing 'uptime' in my linux container shows 5 days, the uptime of the base host.
- justincormack 13y agoIn theory things like the clock could be namespaced too, if anyone found it useful.
- MrBuddyCasino 13y agoThat would be useful - sometimes you have to change the system time to debug some code, and if that trips up all the other containers, its a deal breaker for us.
- djchen 13y agoI was thinking more like FreeBSD jails. Containers can't change the system clock.
- qznc 13y agoNetworking for Docker containers is bridged, so iptables are per container. http://blog.docker.io/2013/08/containers-docker-how-secure-are-they/ http://blog.docker.io/2013/08/containers-docker-how-secure-a...
- mcpherrinm 13y agoLinux network namespaces let you have multiple independent network stacks in linux. That means routing tables, interfaces, routing tables, IP addresses, etc. Each container gets its own network namespace (along with other namespaces like hostname, pids, users, ipc, filesystem mounts). Anything not handled by one of the 6 namespaces is the same across all containers. That includes things like what kernel modules are loaded, the system clock, etc. Because a user with root can manipulate the kernel in many ways, I wouldn't give root to an untrusted user and assume containers were enough to contain them. Certainly if they can load a custom kernel module it's game over, but I'd bet there's plenty of other ways to break out too.
- k3oni 13y agoWelcome back OpenVZ... except that this will be part of the OS now.. and i wouldn't call these "an optimization of virtual machines".. containers are part of the kernel/OS so call it kernel/OS virtualization based if you want where virtual machines are or supposed to be hardware virtualization based. Ex. openvz,lxc - containers kvm,xen - virtual machines
- memracom 13y agoOpenVz was something that required hacked kernels. A better analogy is LXC = Solaris Zones KVM and XEN are hypervisors for virtualization. VirtualBox is a non-hypervisor virtual machine.
- vidarh 13y agoOpenVz is something that requires "hacked kernels" only because the changes were not accepted upstream. I don't think your analogy is any better. We could tack on an " = OpenVz" on there - all three of them are pretty similar.
- diminoten 13y agoExcept when you need snapshotting, live-migration, or have a highly disparate set of underlying OS requirements. The author of the article explicitly says, "This isn't to say that they're going to replace virtual machines." and I agree - there's no point in pretending like they're solving all of the same problems. I guess you could say that containers are an optimization of some use cases of VMs.
- gaius 13y agoBut that's simply not true. 20 years ago you could live migrate processes around a VAXcluster with nary a VM in sight.
- wmf 13y agoBut since history does not necessarily flow in one direction, today you need a hypervisor (or experiment with CRIU) to do live migration.
- ghaff 13y agoAlmost. Although I don't remember the details, IBM has Live Partition Mobility for WPARs on AIX. They were looking for differentiation from Solaris Containers at the time and bought a startup called Meiosys, which had some tech which helped them add live migration to WPARs.
- wmf 13y agoHeh. Even though I work for IBM I wouldn't bother mentioning AIX on HN; it simply doesn't exist inside the echo chamber.
- pigs 13y agoYou actually have to buy an add-on called WPAR Manager[1] to enable live mobility; it's not available in baseline AIX. They have been shipping this tech for several years now. Disclaimer: I was on the WPAR Manager dev team for several releases. [1] http://www-03.ibm.com/systems/power/software/aix/sysmgmt/wpar/ http://www-03.ibm.com/systems/power/software/aix/sysmgmt/wpa...
- zobzu 13y agoThats kinda true indeed. Containers people want to use are VMs that directly talks to the local kernel (i.e. it does not run another kernel on top, like virtualization does). You can make process groups without making the container look like a VM ... BUT. That's hard and time consuming for little gain.
- pjmlp 13y agoThat is why micro-hypervisors where all OS instances are virtualized even the one acting as master, like on mainframes, are gaining acceptance.
- cbsmith 13y agoAs someone who has used OpenVZ for a long time, I'd have to disagree with the notion that containers are simply process groups with access restrictions. You have a distinct namespace for them (including networking), which really changes everything.
- lsc 13y agoGenerally speaking, containers have worse resource separation. Last time I looked, a system with containers shared pagecache. Sharing pagecache means that one user thrashing the disk can make the system unusable for everyone, well, to the extent that no system without pagecache is usable. Xen and KVM mitigate this by giving you your own ram and thus your own pagecache. Personally? I think this makes a way bigger difference than any cgroup limiting of iops. Pagecache is not free ram, and should not be treated as such. I think containers are one of those tradeoffs; they are more efficient when it comes to hardware, but they are less efficient when it comes to sysadmin time tracking down users who are using more resources than they ought. On the other hand, some find managing multiple containers easier than managing multiple complete virtual servers.
- jsmthrowaway 13y agoPart of the problem, Luke, is that you're thinking about containers in terms of a hosting provider. That's not your fault. You're a hosting provider. To you, cotenancy is not only a security concern but also performance and SLA. For almost everyone else, efficiency is way paramount. I still think right now the best way forward is: - Hosting provider: Xen/etc - Internal clusters: containers Containers merely bring commodity computing closer to HPC operational techniques. Containers are not useful for you at prgmr. Using every drop of RAM on the machine for page cache is a positive for my use case, not a negative. Virt overhead makes no sense in internal clusters, in my opinion. I've worked in an all virt environment and an all container environment. The latter is way better, and these are scales where Puppet burning a core for several seconds across the fleet has a demonstrable impact on Opex.
- lsc 13y ago>Part of the problem, Luke, is that you're thinking about containers in terms of a hosting provider. That's not your fault. You're a hosting provider. To you, cotenancy is not only a security concern but also performance and SLA. For almost everyone else, efficiency is way paramount. Oh yeah. More fair is almost always preferable to more efficient when you are in a multi-user situation, and certainly, containers are more efficient. I... have always had a problem, though, with this idea that you should virtualize clustered production applications; virtualization takes a big server and cuts it up into little servers (at a cost) - whereas clustering takes a bunch of little servers and combines them into a big server, again at a cost. It seems to me like once you are out of testing and actually need a full server worth of capacity, you don't want to use virtualization anymore. Multi-tenancy is a feature I need, sure, but for the cluster syadmin? it's all downside. containers/chroots do make a lot more sense than Xen or Kvm if you must virtualize your production clusters; I guess that's the idea behind docker; make a clean way to pass around directories full of binaries. I mean, to be fair, making packages /is/ a pain in the ass, so I can understand wanting a clean way to pass around your production code without using packages. Really, the more I think about it, the more I see containers as an easy way to roll back all the benefits and problems of using shared libraries without going through the incredible pain of statically linking everything. Distribute your "binary" along with all the libraries and the environment it requires. Hah. And actually? I can understand why you would want to do that; Before I was a service provider I was a cluster sysadmin, and I've put in my time resolving shared-library conflicts.