5 ms·

The jails vs containers framing is interesting but I think it misses why Docker actually won. It wasn't the isolation tech. It was the ecosystem: Dockerfiles as

The jails vs containers framing is interesting but I think it misses why Docker actually won. It wasn't the isolation tech. It was the ecosystem: Dockerfiles as executable documentation, a public registry, and compose for local dev. You could pull an image and have something running in 30 seconds without understanding anything about cgroups or namespaces. FreeBSD jails were technically solid years before Docker existed, but the onboarding story was rough. You needed to understand the FreeBSD base system first. Docker let you skip all of that. That said, I've been seeing more people question the container stack complexity recently. Especially for smaller deployments where a jail or even a plain VM with good config management would be simpler and more debuggable. The pendulum might be swinging back a bit for certain use cases.

sthuck 8mo ago
I don't think article misses it, it's exactly the point it makes
torstenvl 8mo ago
> Jails solve the isolation problem beautifully, but they don't have a native answer to shipping. That gap is real, and it's one of the main reasons the ecosystem around jails feels underdeveloped compared to Docker's world. The link literally uses the term ecosystem. Several times actually.
steve1977 8mo ago
> You could pull an image and have something running in 30 seconds without understanding anything Fixed that for you ;)
steve1977 8mo ago
Maybe FreeBSD doesn't want a jails "ecosystem"?
wolvoleo 8mo ago
Jails were never going to 'win' because they're only on an OS with 0.1% marketshare. But it's not a competition. FreeBSD does its thing and Linux does another. That's why I use FreeBSD.
- aswanson 8mo ago
  What is your use case for BSD?
  - gtech1 8mo ago
    *everything. I've really been using it since 4.x. Imagine this: being able to upgrade a system in-place with freebsd-update from minor to major to minor version without everything breaking or having to say a prayer before. And that's just one thing I love about it. Clear separation of userland (/usr/local/etc), rock-solid stability in networking, zfs on root. I had to do 'bonded' interfaces on Debian the other day. It's what, 5 different config files depending on which 'network manager' you use. In FreeBSD it's 5 lines in /etc/rc.conf and you're done. And don't even get me started on betting which distribution (ahem CentOS) will go away next.
    - irishcoffee 8mo ago
      Centos didn’t go away. It changed. Rocky (et. al.) took the old centos role, and I see this as a win/win for everybody. Ubuntu is the disaster Linux distro, I won’t touch Ubuntu if I have any other option.
      - gtech1 8mo ago
        I actually laughed out loud. Try upgrading CentOS to Rocky vs FreeBSD 11 to 15 ( that's FOUR major versions from 2017 I think ), and tell me again how good it is. In LTS environments where I need to upgrade OS's, FreeBSD is a no-brainer.
        irishcoffee 8mo ago
        > I actually laughed out loud. Try upgrading CentOS to Rocky vs FreeBSD 11 to 15 ( that's FOUR major versions from 2017 I think ), and tell me again how good it is. I laughed out loud, there is no in-place upgrade mechanism for that in those distros and never has been, that is the nature of those distros. They release patch/security updates until they go EOL, which is measured in units closer to decades than years. I don’t have a problem with BSDs. That’s cool you like upgrading in place. The best and most laugh-inducing part of your whole point is that centos now not only allows you to do in-place upgrades, that’s the whole fucking point.
chuckadams 8mo ago
Docker's client/server design also allowed for things like Docker Desktop, which made the integration seamless with non-linux systems. Jails have nothing like that, so the only system that will ever run jails is FreeBSD. Also, I'm not up to speed enough to know, but do jails even have a concept of container images?
- Gud 8mo ago
  It’s just files on the filesystem. So tar for imaging?
  - chuckadams 8mo ago
    Plus a script to unpack the tarball somewhere and launch some entry point in a jail. Not conceptually hard, but the OCI spec has a bit more to it than that, and now we're into "write dropbox with rsync" territory... I did some looking around, and I see that ocijail is a thing, so that's probably what I was looking for. (edited, sorry, I didn't see your reply)
    - Gud 8mo ago
      What do you mean”launch an entry point”? The rc script would naturally be included.
user3939382 8mo ago
You can also run Linux containers on FreeBSD https://youtu.be/HV-wUUzRCMo https://youtu.be/HV-wUUzRCMo
- sidkshatriya 8mo ago
  I've tried this ... I've haven't got much mileage on this, sadly. Many Linux syscalls are unemulated and things like /proc/<pid>/fd/NN etc are not "magic symlinks" like on Linux so execve on them fails (i.e there is rudimentary /proc support, it's not full fleshed out). TL;DR Linux containers on FreeBSD via the podman + linuxulator feel half baked. For example, try using the alpine container... `apk upgrade` will fail due to the /proc issue discussed above. Try using the Fedora container `dnf upgrade` will fail due to some seccomp issue. The future of containers on FreeBSD is FreeBSD OCI containers, not (emulated) Linux containers. As an aside, podman on FreeBSD requires sudo which kinda defeats the concept but hopefully this will be fixed in the future.
KronisLV 8mo ago
> the container stack complexity I'm using either Docker Compose or Docker Swarm without Kubernetes, and there's not that much of it, to be honest. My "ingress" is just an Apache2 container that's bound to 80/443 and my storage is either volumes or bind mounts, with no need for more complexity there. > The jails vs containers framing is interesting but I think it misses why Docker actually won. It wasn't the isolation tech. It was the ecosystem: Dockerfiles as executable documentation, a public registry, and compose for local dev. You could pull an image and have something running in 30 seconds without understanding anything about cgroups or namespaces. So where's Jailsfiles? Where's Jail Hub (maybe naming needs a bit of work)? Where's Jail Desktop or Jail Compose or Jail Swarm or Jailbernetes? It feels like either the people behind the various BSDs don't care much for what allowed Docker to win, or they're unable to compete with it, which is a shame, because it'd probably be somewhere between a single and double digit percent userbase growth if they decided to do it and got it right. They already have some of the foundational tech, so why not the UX and the rest of it?
- jacquesm 8mo ago
  > I'm using either Docker Compose or Docker Swarm without Kubernetes, and there's not that much of it, to be honest. On the outside. But that's a lot of complexity hidden from view there, easily a couple of million lines of code on top of the code that you wrote.
- whizzter 8mo ago
  I think Jails started as a tool of it's time, it's about the same thing as virtualization in making isolated systems when dependencies start to diverge, but aimed at the issues of sysadmins that had to manage their own systems, not a quick developer experience. Even if "jailsfiles" were created the ecosystem would need to start from scratch and sometimes it feels like people in the FreeBSD ecosystem have a hard enough time keeping ports somewhat up to date, let alone create something new. Luckily Podman seems to support FreeBSD these days for docker images, but the Linux emualation might be a bit of a blocker so not a 100% solution.
- jcgrillo 8mo ago
  I never used this, but noticed it in some docs back when I was using Nomad and thought it was an intriguing idea: https://github.com/cneira/jail-task-driver https://github.com/cneira/jail-task-driver

sthuck 8mo ago

I don't think article misses it, it's exactly the point it makes

torstenvl 8mo ago

> Jails solve the isolation problem beautifully, but they don't have a native answer to shipping. That gap is real, and it's one of the main reasons the ecosystem around jails feels underdeveloped compared to Docker's world. The link literally uses the term ecosystem. Several times actually.

steve1977 8mo ago

> You could pull an image and have something running in 30 seconds without understanding anything Fixed that for you ;)

steve1977 8mo ago

Maybe FreeBSD doesn't want a jails "ecosystem"?

wolvoleo 8mo ago

Jails were never going to 'win' because they're only on an OS with 0.1% marketshare. But it's not a competition. FreeBSD does its thing and Linux does another. That's why I use FreeBSD.

aswanson 8mo ago
What is your use case for BSD?
- gtech1 8mo ago
  *everything. I've really been using it since 4.x. Imagine this: being able to upgrade a system in-place with freebsd-update from minor to major to minor version without everything breaking or having to say a prayer before. And that's just one thing I love about it. Clear separation of userland (/usr/local/etc), rock-solid stability in networking, zfs on root. I had to do 'bonded' interfaces on Debian the other day. It's what, 5 different config files depending on which 'network manager' you use. In FreeBSD it's 5 lines in /etc/rc.conf and you're done. And don't even get me started on betting which distribution (ahem CentOS) will go away next.
  - irishcoffee 8mo ago
    Centos didn’t go away. It changed. Rocky (et. al.) took the old centos role, and I see this as a win/win for everybody. Ubuntu is the disaster Linux distro, I won’t touch Ubuntu if I have any other option.
    - gtech1 8mo ago
      I actually laughed out loud. Try upgrading CentOS to Rocky vs FreeBSD 11 to 15 ( that's FOUR major versions from 2017 I think ), and tell me again how good it is. In LTS environments where I need to upgrade OS's, FreeBSD is a no-brainer.
      - irishcoffee 8mo ago
        > I actually laughed out loud. Try upgrading CentOS to Rocky vs FreeBSD 11 to 15 ( that's FOUR major versions from 2017 I think ), and tell me again how good it is. I laughed out loud, there is no in-place upgrade mechanism for that in those distros and never has been, that is the nature of those distros. They release patch/security updates until they go EOL, which is measured in units closer to decades than years. I don’t have a problem with BSDs. That’s cool you like upgrading in place. The best and most laugh-inducing part of your whole point is that centos now not only allows you to do in-place upgrades, that’s the whole fucking point.

chuckadams 8mo ago

Docker's client/server design also allowed for things like Docker Desktop, which made the integration seamless with non-linux systems. Jails have nothing like that, so the only system that will ever run jails is FreeBSD. Also, I'm not up to speed enough to know, but do jails even have a concept of container images?

Gud 8mo ago
It’s just files on the filesystem. So tar for imaging?
- chuckadams 8mo ago
  Plus a script to unpack the tarball somewhere and launch some entry point in a jail. Not conceptually hard, but the OCI spec has a bit more to it than that, and now we're into "write dropbox with rsync" territory... I did some looking around, and I see that ocijail is a thing, so that's probably what I was looking for. (edited, sorry, I didn't see your reply)
  - Gud 8mo ago
    What do you mean”launch an entry point”? The rc script would naturally be included.

user3939382 8mo ago

You can also run Linux containers on FreeBSD https://youtu.be/HV-wUUzRCMo https://youtu.be/HV-wUUzRCMo

sidkshatriya 8mo ago
I've tried this ... I've haven't got much mileage on this, sadly. Many Linux syscalls are unemulated and things like /proc/<pid>/fd/NN etc are not "magic symlinks" like on Linux so execve on them fails (i.e there is rudimentary /proc support, it's not full fleshed out). TL;DR Linux containers on FreeBSD via the podman + linuxulator feel half baked. For example, try using the alpine container... `apk upgrade` will fail due to the /proc issue discussed above. Try using the Fedora container `dnf upgrade` will fail due to some seccomp issue. The future of containers on FreeBSD is FreeBSD OCI containers, not (emulated) Linux containers. As an aside, podman on FreeBSD requires sudo which kinda defeats the concept but hopefully this will be fixed in the future.

KronisLV 8mo ago

> the container stack complexity I'm using either Docker Compose or Docker Swarm without Kubernetes, and there's not that much of it, to be honest. My "ingress" is just an Apache2 container that's bound to 80/443 and my storage is either volumes or bind mounts, with no need for more complexity there. > The jails vs containers framing is interesting but I think it misses why Docker actually won. It wasn't the isolation tech. It was the ecosystem: Dockerfiles as executable documentation, a public registry, and compose for local dev. You could pull an image and have something running in 30 seconds without understanding anything about cgroups or namespaces. So where's Jailsfiles? Where's Jail Hub (maybe naming needs a bit of work)? Where's Jail Desktop or Jail Compose or Jail Swarm or Jailbernetes? It feels like either the people behind the various BSDs don't care much for what allowed Docker to win, or they're unable to compete with it, which is a shame, because it'd probably be somewhere between a single and double digit percent userbase growth if they decided to do it and got it right. They already have some of the foundational tech, so why not the UX and the rest of it?

jacquesm 8mo ago
> I'm using either Docker Compose or Docker Swarm without Kubernetes, and there's not that much of it, to be honest. On the outside. But that's a lot of complexity hidden from view there, easily a couple of million lines of code on top of the code that you wrote.
whizzter 8mo ago
I think Jails started as a tool of it's time, it's about the same thing as virtualization in making isolated systems when dependencies start to diverge, but aimed at the issues of sysadmins that had to manage their own systems, not a quick developer experience. Even if "jailsfiles" were created the ecosystem would need to start from scratch and sometimes it feels like people in the FreeBSD ecosystem have a hard enough time keeping ports somewhat up to date, let alone create something new. Luckily Podman seems to support FreeBSD these days for docker images, but the Linux emualation might be a bit of a blocker so not a 100% solution.
jcgrillo 8mo ago
I never used this, but noticed it in some docs back when I was using Nomad and thought it was an intriguing idea: https://github.com/cneira/jail-task-driver https://github.com/cneira/jail-task-driver