5 ms·

IMO, poor marketing and terrible UX. It's the same reason Solaris zones never got popular. I tried to use zones many years ago.. For "isolated services" I mo

IMO, poor marketing and terrible UX. It's the same reason Solaris zones never got popular. I tried to use zones many years ago.. For "isolated services" I mostly ran things like debian VMs with 128M of ram and 4G of disk space. In theory zones could do that too, but solaris was fundamentally incompatible with the idea of running a tiny 'container' with only the software that you needed installed.

laumars 11y ago
OS containers will require more than just the application you want sandboxed. This is true for Solaris Zones, FreeBSD Jail, Linux OpenVZ or LXC (Linux Containers). And that's fine because the concept is akin to running a VM across a shared kernel rather than abstracting the hardware via a hypervisor. So you're effectively chrooting a sub-OS inside the host. As for your comments regarding Solaris, I was using Solaris Zones about a decade ago and, inversely to your experience, I found the whole process really easy. In fact it was on Solaris where I first learned about OS containers and which really convinced me to adopt them to many other projects over the years since. To be honest, it is really weird reading your comment as my experiences couldn't differ more. If your memory is up to the job, I'd be curious to read more detail about the issues you had.
- justinsaccount 11y ago
  Sure.. Creating a zone was easy, it was managing it that was impossible. So the way I used to use debian was I was using xen. If I needed something like a tftp server, I would bring up a tiny VM with nothing but the base debian system installed. Then I would simply apt-get install tftp-server-whatever (via puppet) and call it a day. Since all it was running was sshd+syslog+postfix(bound to lo) and the tftp server, patching was easy. I didn't have to deal with managing updates for 2000 packages that I was not using. I could also upgrade each vm separately. Zones did not work that way at all. You could do a 'sparse' zone, which would inherit some directories from the host system, or whatever the opposite of that was that would just go and install a full 4G+ solaris system into the zone. When I complained about the fact 'single use' zones had to have 4G of software installed, I was told to just use clones, which did not use extra disk space. The problem is those full clones then needed to be patched. It was basically impossible to create a small single purpose zone that did not contain 4G of software and was easy to manage. The sparse zones were not usable because then you could not install or update software in isolation. I see the same thing on the ez-jail page: > Because the basejail's copy of the userland is shared by the other jails, updating the basejail automatically updates all of the other jails. Either source or binary updates can be used. So, if an update to basejail breaks something, it breaks every jail linked to it? What if I want to test my software on FreeBSD+1? Really, the problem wasn't the zone technology, the problem was the horrible user-hostile solaris userland. This is why Triton appeals to people.
  - cm3 11y ago
    Triton?
    - res0nat0r 11y ago
      http://www.joyent.com http://www.joyent.com
  - ZenoArrow 11y ago
    Triton looks interesting, thanks for the tip.
  - laumars 11y ago
    > "The problem is those full clones then needed to be patched." As would the VM's you said you ran previously. Sadly patching is one of the biggest problems in IT. Which is where tools like Puppet come into their own (which I think you did say you run?). > "The sparse zones were not usable because then you could not install or update software in isolation. I see the same thing on the ez-jail page: > > "Because the basejail's copy of the userland is shared by the other jails, updating the basejail automatically updates all of the other jails. Either source or binary updates can be used." I think that's a little deceptive. You can still install packages in the containers separate from the basejail. ez-jails would be completely useless if you couldn't. basejail is essentially just the FreeBSD core (ie it excludes /usr/local). > "So, if an update to basejail breaks something, it breaks every jail linked to it? What if I want to test my software on FreeBSD+1?" Then you'd have that as a container without basejails inheritance. Nobody says you have to create every jail using the same templates (though you did earlier complain about having to patch everything in isolation :P) Personally, I tend to just take a ZFS snapshot and then rollback if I ever run into issues. It's a little lazy but works well enough for planned maintenance windows. ----- Those arguments aside you do raise some interesting points; and I hadn't heard of Triton either. So I've definitely learned something new today. I can't really say I've had the same requirements to spin up and destroy new containers with the same frequency that you have, which would explain the differences between our experiences. For what it's worth, ez-jail does have a solution to those kinds of usages via "flavours" (loosely like VMWare VM templates). However I suspect this might still fall a little short of your specific requirements.
- toast0 11y ago
  FreeBSD jails can absolute have just the application you want sandboxed (and its config, datafiles, and linked libraries, and the library loader), at work I have a couple jails like that. We also have jails that have a full os. It's not as obvious or well documented how to set it up this way, but it was maybe a couple days work, including my first rc script ever, and the script to assemble the jail in our environment (you could probably have a tar file with the contents of the jail, and just add the conf/data files)
  - laumars 11y ago
    Did you document your process? I'd love to read about it
    - toast0 11y ago
      Did not document (other than the assembly script that I can't share), although I did post my jail.conf (essentially) in an earlier FreeBSD thread https://news.ycombinator.com/item?id=10602990 https://news.ycombinator.com/item?id=10602990 If you know how to build a minimal chroot, you know how to build a minimal jail, and then you just need to configure it, etc... If you don't know how to build a minimal chroot, check out option 3 'most minimal chroot possible' here: http://ram.kossboss.com/debian-ubuntu-build-minimal-chroot/ http://ram.kossboss.com/debian-ubuntu-build-minimal-chroot/ It's written against Debian Linux, but the same thing basically works for FreeBSD, except ldd on FreeBSD omits the loader (/libexec/ld-elf.so.1), but I sort of remembered that from somewhere, and I used truss to see what system calls were being used before it died.

laumars 11y ago

OS containers will require more than just the application you want sandboxed. This is true for Solaris Zones, FreeBSD Jail, Linux OpenVZ or LXC (Linux Containers). And that's fine because the concept is akin to running a VM across a shared kernel rather than abstracting the hardware via a hypervisor. So you're effectively chrooting a sub-OS inside the host. As for your comments regarding Solaris, I was using Solaris Zones about a decade ago and, inversely to your experience, I found the whole process really easy. In fact it was on Solaris where I first learned about OS containers and which really convinced me to adopt them to many other projects over the years since. To be honest, it is really weird reading your comment as my experiences couldn't differ more. If your memory is up to the job, I'd be curious to read more detail about the issues you had.

justinsaccount 11y ago
Sure.. Creating a zone was easy, it was managing it that was impossible. So the way I used to use debian was I was using xen. If I needed something like a tftp server, I would bring up a tiny VM with nothing but the base debian system installed. Then I would simply apt-get install tftp-server-whatever (via puppet) and call it a day. Since all it was running was sshd+syslog+postfix(bound to lo) and the tftp server, patching was easy. I didn't have to deal with managing updates for 2000 packages that I was not using. I could also upgrade each vm separately. Zones did not work that way at all. You could do a 'sparse' zone, which would inherit some directories from the host system, or whatever the opposite of that was that would just go and install a full 4G+ solaris system into the zone. When I complained about the fact 'single use' zones had to have 4G of software installed, I was told to just use clones, which did not use extra disk space. The problem is those full clones then needed to be patched. It was basically impossible to create a small single purpose zone that did not contain 4G of software and was easy to manage. The sparse zones were not usable because then you could not install or update software in isolation. I see the same thing on the ez-jail page: > Because the basejail's copy of the userland is shared by the other jails, updating the basejail automatically updates all of the other jails. Either source or binary updates can be used. So, if an update to basejail breaks something, it breaks every jail linked to it? What if I want to test my software on FreeBSD+1? Really, the problem wasn't the zone technology, the problem was the horrible user-hostile solaris userland. This is why Triton appeals to people.
- cm3 11y ago
  Triton?
  - res0nat0r 11y ago
    http://www.joyent.com http://www.joyent.com
- ZenoArrow 11y ago
  Triton looks interesting, thanks for the tip.
- laumars 11y ago
  > "The problem is those full clones then needed to be patched." As would the VM's you said you ran previously. Sadly patching is one of the biggest problems in IT. Which is where tools like Puppet come into their own (which I think you did say you run?). > "The sparse zones were not usable because then you could not install or update software in isolation. I see the same thing on the ez-jail page: > > "Because the basejail's copy of the userland is shared by the other jails, updating the basejail automatically updates all of the other jails. Either source or binary updates can be used." I think that's a little deceptive. You can still install packages in the containers separate from the basejail. ez-jails would be completely useless if you couldn't. basejail is essentially just the FreeBSD core (ie it excludes /usr/local). > "So, if an update to basejail breaks something, it breaks every jail linked to it? What if I want to test my software on FreeBSD+1?" Then you'd have that as a container without basejails inheritance. Nobody says you have to create every jail using the same templates (though you did earlier complain about having to patch everything in isolation :P) Personally, I tend to just take a ZFS snapshot and then rollback if I ever run into issues. It's a little lazy but works well enough for planned maintenance windows. ----- Those arguments aside you do raise some interesting points; and I hadn't heard of Triton either. So I've definitely learned something new today. I can't really say I've had the same requirements to spin up and destroy new containers with the same frequency that you have, which would explain the differences between our experiences. For what it's worth, ez-jail does have a solution to those kinds of usages via "flavours" (loosely like VMWare VM templates). However I suspect this might still fall a little short of your specific requirements.
toast0 11y ago
FreeBSD jails can absolute have just the application you want sandboxed (and its config, datafiles, and linked libraries, and the library loader), at work I have a couple jails like that. We also have jails that have a full os. It's not as obvious or well documented how to set it up this way, but it was maybe a couple days work, including my first rc script ever, and the script to assemble the jail in our environment (you could probably have a tar file with the contents of the jail, and just add the conf/data files)
- laumars 11y ago
  Did you document your process? I'd love to read about it
  - toast0 11y ago
    Did not document (other than the assembly script that I can't share), although I did post my jail.conf (essentially) in an earlier FreeBSD thread https://news.ycombinator.com/item?id=10602990 https://news.ycombinator.com/item?id=10602990 If you know how to build a minimal chroot, you know how to build a minimal jail, and then you just need to configure it, etc... If you don't know how to build a minimal chroot, check out option 3 'most minimal chroot possible' here: http://ram.kossboss.com/debian-ubuntu-build-minimal-chroot/ http://ram.kossboss.com/debian-ubuntu-build-minimal-chroot/ It's written against Debian Linux, but the same thing basically works for FreeBSD, except ldd on FreeBSD omits the loader (/libexec/ld-elf.so.1), but I sort of remembered that from somewhere, and I used truss to see what system calls were being used before it died.