10 ms·

Google has plenty of experience with containers already, since they heavily use cgroups and the concept of containers in their production environment for isolat

Google has plenty of experience with containers already, since they heavily use cgroups and the concept of containers in their production environment for isolation and resource control, never mind the fact that two of their engineers had written much of the initial cgroups code. I talked about this in my "Introduction to Containers on Linux using LXC" screencast [1]. Briefly, Google is in the process of open sourcing their internal container code [2], there was a Wired article that talked about their container orchestration system [3], and finally, there was John Wilkes (Google Cluster Management), who talks about the container management system [4]. Docker adds very interesting filesystem ideas and software for the management of container images. Personally, I think we are on the cusp of a transition from VPS (xen/hvm) to VPS (containers). I also hope that Google throws some of their concepts at the Docker project. Interesting times for this space. [1] http://sysadmincasts.com/episodes/24-introduction-to-containers-on-linux-using-lxc http://sysadmincasts.com/episodes/24-introduction-to-contain... [2] https://github.com/google/lmctfy https://github.com/google/lmctfy [3] http://www.wired.com/2013/03/google-borg-twitter-mesos/all/ http://www.wired.com/2013/03/google-borg-twitter-mesos/all/ [4] http://www.youtube.com/watch?v=0ZFMlO98Jkc http://www.youtube.com/watch?v=0ZFMlO98Jkc

tzm 12y ago
Great points. Btw, Victor and Rohit from the Google LMCTFY team are now active maintainers of libcontainer. https://github.com/dotcloud/docker/blob/master/pkg/libcontainer/MAINTAINERS https://github.com/dotcloud/docker/blob/master/pkg/libcontai... Also check out Joe Beda's deck from GlueCon: http://slides.eightypercent.net/GlueCon%202014%20-%20Containers%20At%20Scale.pdf http://slides.eightypercent.net/GlueCon%202014%20-%20Contain... Docker is a natural fit for GCE.
- WestCoastJustin 12y ago
  Great links -- thanks!
jacquesm 12y ago
> Personally, I think we are on the cusp of a transition from VPS (xen/hvm) to VPS (containers). I'm not so sure of that. I think a lot of the use-cases for VMs are based on isolation between users and making sure everybody gets a fair slice. Something like docker would work well with a single tenant but for multi-tenant usage docker would give you all the headaches of a shared host and very little of the benefits of a VM. For those use cases you're probably going to see multiple docker instances for a single tenant riding on top of a VM. The likes of Heroku, AWS, Google etc will likely use docker or something very much like it as a basic unit to talk to their customers, but underneath it they'll (hopefully) be walling off tenants with VMs first. VMs don't have to play friendly with each other, docker containers likely will have to behave nicely if they're not to monopolize the underlying machine.
- jamesaguilar 12y ago
  > I think a lot of the use-cases for VMs are based on isolation between users and making sure everybody gets a fair slice. Containers do this.
  - jacquesm 12y ago
    Hm. We'll see about that. I can see a whole pile of potential issues here with 'breaking out of the docker' on par with escaping from the sandbox and breaking the chroot jail, which I see this as a luxury version of. Of course you could try to escalate from a VM to the host (see cloudburst) but that's a rarity. Docker seems to be less well protected against that sort of thing, but I'm nowhere near qualified to make that evaluation so I'll stick to 'seems' for now. It looks like the jump is a smaller one than from a VM.
    - thockingoog 12y ago
      Fair usage of resources and security isolation are two VERY different problems. Containers can be VERY good at resource isolation. Security has not really been figured out yet.
    - jamesaguilar 12y ago
mikecb 12y ago
There's also the recent article about automatic and machine-learning based power and core management.[1], [2] If anyone here specializes in similar things, I would be curious to know if this Pegasus system runs on top of or underneath Borg/Omega (or perhaps replaced it?), or is a separate system altogether. [1] http://www.theregister.co.uk/2014/06/05/google_pegasus_system http://www.theregister.co.uk/2014/06/05/google_pegasus_syste... Edit: [2] http://gigaom.com/2014/05/28/google-is-harnessing-machine-learning-to-cut-data-center-energy http://gigaom.com/2014/05/28/google-is-harnessing-machine-le...
billybofh 12y ago
I'm always surprised that OpenVZ[1] doesn't come up more in discussions about containers. I used to use it extensively in for small lightweight VPS's (usually just an apache or mysql instance) and always found it to be pretty impressive. I've used Docker to make a debian-packaged app available on a bunch of CentOS machines and it saved me a huge headache (the dependency tree for the app was huge) so I'm a fan - but still a little puzzled at OpenVZ's seeming invisibility. [1] http://openvz.org/Main_Page http://openvz.org/Main_Page
- wmf 12y ago
  OpenVZ was basically the prototype for LXC. Distros seem to have better support for LXC since it's "official".
  - billybofh 12y ago
    Yeah, I realise it's place in LXC history. It still seems slightly odd that it's been kind of overlooked. It offered quite a lot (and still does) that I don't see replicated in any of the other container packages. At least not without quite a lot of manual faff. Possibly it was just a little ahead of it's time and was also overshadowed by the rise of HW virtualisation in the later 2000's. Having to install a custom kernel (certainly when I used it) was also a bit of a hassle mind you. Anyway - maybe someone will re-invent the toolchain using Swift or Node and it'll become cool again ;-)
    - justincormack 12y ago
      It also has a security focus, while docker has started with convenience for deployment rather than looking like an isolated machine.
  - contingencies 12y ago
    Don't forget Linux vServers as well.
- FooBarWidget 12y ago
  It's probably because of the way OpenVZ is marketed (or should I say, not marketed). OpenVZ's technology could probably do the same as what Docker does but they're not marketing it in the same way. The concept matters just as much as the actual technology.
dragonwriter 12y ago
> Personally, I think we are on the cusp of a transition from VPS (xen/hvm) to VPS (containers). There may be some of that, but I think more common will be continuing to have tradition IaaS bring-your-OS-image services, with a new tier in between IaaS and PaaS (call it CaaS -- Container host as a service), plus a common use of IaaS being to deploy an OS that works largely as a container host (something like CoreOS).
rsync 12y ago
"Personally, I think we are on the cusp of a transition from VPS (xen/hvm) to VPS (containers)." A transition back to, I think ... the very first VPS provider (JohnCompanies, 2001)[1] was based entirely on FreeBSD jail and was entirely container based, even though those containers were presented as standalone FreeBSD systems. [1] Yes, verio did have that odd VPS-like enterprise service earlier than that, but JC did the first VPS as we came to know it.
- Touche 12y ago
  Pardon my ignorance but don't containers share the same kernel as the host? Meaning I can't run a Ubuntu container in a BSD jail or vice-versa? I don't want to use containers if it limits my OS choice to being from the same family as the host.

tzm 12y ago

Great points. Btw, Victor and Rohit from the Google LMCTFY team are now active maintainers of libcontainer. https://github.com/dotcloud/docker/blob/master/pkg/libcontainer/MAINTAINERS https://github.com/dotcloud/docker/blob/master/pkg/libcontai... Also check out Joe Beda's deck from GlueCon: http://slides.eightypercent.net/GlueCon%202014%20-%20Containers%20At%20Scale.pdf http://slides.eightypercent.net/GlueCon%202014%20-%20Contain... Docker is a natural fit for GCE.

WestCoastJustin 12y ago
Great links -- thanks!

jacquesm 12y ago

> Personally, I think we are on the cusp of a transition from VPS (xen/hvm) to VPS (containers). I'm not so sure of that. I think a lot of the use-cases for VMs are based on isolation between users and making sure everybody gets a fair slice. Something like docker would work well with a single tenant but for multi-tenant usage docker would give you all the headaches of a shared host and very little of the benefits of a VM. For those use cases you're probably going to see multiple docker instances for a single tenant riding on top of a VM. The likes of Heroku, AWS, Google etc will likely use docker or something very much like it as a basic unit to talk to their customers, but underneath it they'll (hopefully) be walling off tenants with VMs first. VMs don't have to play friendly with each other, docker containers likely will have to behave nicely if they're not to monopolize the underlying machine.

jamesaguilar 12y ago
> I think a lot of the use-cases for VMs are based on isolation between users and making sure everybody gets a fair slice. Containers do this.
- jacquesm 12y ago
  Hm. We'll see about that. I can see a whole pile of potential issues here with 'breaking out of the docker' on par with escaping from the sandbox and breaking the chroot jail, which I see this as a luxury version of. Of course you could try to escalate from a VM to the host (see cloudburst) but that's a rarity. Docker seems to be less well protected against that sort of thing, but I'm nowhere near qualified to make that evaluation so I'll stick to 'seems' for now. It looks like the jump is a smaller one than from a VM.
  - thockingoog 12y ago
    Fair usage of resources and security isolation are two VERY different problems. Containers can be VERY good at resource isolation. Security has not really been figured out yet.
  - jamesaguilar 12y ago

mikecb 12y ago

There's also the recent article about automatic and machine-learning based power and core management.[1], [2] If anyone here specializes in similar things, I would be curious to know if this Pegasus system runs on top of or underneath Borg/Omega (or perhaps replaced it?), or is a separate system altogether. [1] http://www.theregister.co.uk/2014/06/05/google_pegasus_system http://www.theregister.co.uk/2014/06/05/google_pegasus_syste... Edit: [2] http://gigaom.com/2014/05/28/google-is-harnessing-machine-learning-to-cut-data-center-energy http://gigaom.com/2014/05/28/google-is-harnessing-machine-le...

billybofh 12y ago

I'm always surprised that OpenVZ[1] doesn't come up more in discussions about containers. I used to use it extensively in for small lightweight VPS's (usually just an apache or mysql instance) and always found it to be pretty impressive. I've used Docker to make a debian-packaged app available on a bunch of CentOS machines and it saved me a huge headache (the dependency tree for the app was huge) so I'm a fan - but still a little puzzled at OpenVZ's seeming invisibility. [1] http://openvz.org/Main_Page http://openvz.org/Main_Page

wmf 12y ago
OpenVZ was basically the prototype for LXC. Distros seem to have better support for LXC since it's "official".
- billybofh 12y ago
  Yeah, I realise it's place in LXC history. It still seems slightly odd that it's been kind of overlooked. It offered quite a lot (and still does) that I don't see replicated in any of the other container packages. At least not without quite a lot of manual faff. Possibly it was just a little ahead of it's time and was also overshadowed by the rise of HW virtualisation in the later 2000's. Having to install a custom kernel (certainly when I used it) was also a bit of a hassle mind you. Anyway - maybe someone will re-invent the toolchain using Swift or Node and it'll become cool again ;-)
  - justincormack 12y ago
    It also has a security focus, while docker has started with convenience for deployment rather than looking like an isolated machine.
- contingencies 12y ago
  Don't forget Linux vServers as well.
FooBarWidget 12y ago
It's probably because of the way OpenVZ is marketed (or should I say, not marketed). OpenVZ's technology could probably do the same as what Docker does but they're not marketing it in the same way. The concept matters just as much as the actual technology.

dragonwriter 12y ago

> Personally, I think we are on the cusp of a transition from VPS (xen/hvm) to VPS (containers). There may be some of that, but I think more common will be continuing to have tradition IaaS bring-your-OS-image services, with a new tier in between IaaS and PaaS (call it CaaS -- Container host as a service), plus a common use of IaaS being to deploy an OS that works largely as a container host (something like CoreOS).

rsync 12y ago

"Personally, I think we are on the cusp of a transition from VPS (xen/hvm) to VPS (containers)." A transition back to, I think ... the very first VPS provider (JohnCompanies, 2001)[1] was based entirely on FreeBSD jail and was entirely container based, even though those containers were presented as standalone FreeBSD systems. [1] Yes, verio did have that odd VPS-like enterprise service earlier than that, but JC did the first VPS as we came to know it.

Touche 12y ago
Pardon my ignorance but don't containers share the same kernel as the host? Meaning I can't run a Ubuntu container in a BSD jail or vice-versa? I don't want to use containers if it limits my OS choice to being from the same family as the host.