5 ms·

The issue I have with Jails, over something like docker, is that the kernel ABI changes between major releases so jails need to migrate with base systems upgrad

The issue I have with Jails, over something like docker, is that the kernel ABI changes between major releases so jails need to migrate with base systems upgrades. It also means a container registry would need to store container versions per release. I like the isolation for my own bespoke home network, but does it work well in production set up’s?

inferiorhuman 3y ago
Kernels are generally backwards compatible and major versions are explicitly supported for five (5) years. Over in Linux land you'll see builds for the different libcs and it seems to work just fine.
- johncolanduoni 3y ago
  The difference is that the container libc version and kernel version don't have to correspond to each other in any way on Linux (unless your kernel is like 20 years older, in which case it wouldn't support Docker anyway).
  - inferiorhuman 3y ago
    The end result is that you're matching the executable to the libc variant instead of to the kernel. In practice it's not much of a difference, if at all. If you need to use older binaries, install the compatibility shims (or use a base jail based on the older version of FreeBSD). In terms of a support matrix there are two major versions of FreeBSD that are given active support, which is less than over in Linux land. All in all of all the concerns I'd have about using jails in a production environment, dealing with different major versions is not on that list.
    - monocasa 3y ago
      But the half the point of the container is per application libcs/userlands, so the kernel ABI is the only required constant.
      - inferiorhuman 3y ago
        In FreeBSD the userland and kernel move in unison, and with the backwards compatibility there's nothing stopping you from running a jail based on a previous version of FreeBSD. The compatX packages are only there to provide the older libraries if you want to run an older application on a newer jail/install. As was pointed out, you can run binaries from a FreeBSD version (4) that's probably older than most posters here. Still not seeing what the big fuss is about.
        monocasa 3y ago
        the hangup is because half the point is to be able to take a random userland and run it on your current system without changing it. As soon as your userland bitrots because the kernel ABI changed, you've lost a lot of the benefit. And to be clear, you can not take a complete userland from 4.x. You have to reinstall your application on a newer userland with compat shims when you change the kernel. Ironically enough, the Linux syscall compat layer in FreeBSD may give you more stability here.
        inferiorhuman 3y ago
        You have to reinstall your application on a newer userland with compat shims when you change the kernel. No, you don't. You can absolutely take a jail based on a previous version of FreeBSD and run it on a newer version. I've not tried going as far back as 4.x, but I've done jail images of 9 and 10 on 11 and 12 hosts as part of a CI system. The shims you're talking about are not shims at all, just packages of the older libraries that a program built for an older version of FreeBSD would expect. You can go ahead and look at the package manifests if you don't believe me. They're literally just a handful of older shared libraries. Honestly, I can't remember a time when the syscall table changed in a backwards incompatible manner. The compat packages are only needed if you're trying to run an older program with a newer userland. If you're running an older jail in a newer freebsd host, you can run your program as-is because those libraries have already been installed.
        3y ago
  - bandrami 3y ago
    I thought that too, until the syscall semantics changed between RHEL 6 and RHEL 7. Spent way more time debugging that than I wanted to.
throw0101a 3y ago
There are compat libraries that are available, so if you are running 13.x (the most current major release) but want to run binaries originally compiled under 12.x, you install: * https://www.freshports.org/misc/compat12x/ https://www.freshports.org/misc/compat12x/ See further: * https://www.freshports.org/misc/compat11x/ https://www.freshports.org/misc/compat11x/ * https://www.freshports.org/misc/compat10x/ https://www.freshports.org/misc/compat10x/ […] * https://www.freshports.org/misc/compat4x/ https://www.freshports.org/misc/compat4x/
- nucleardog 3y ago
  And for anyone that isn't intimately familiar with FreeBSD's release history or anything, I feel like it's worth pointing out that the 4.x major release dates from over 23 years ago. You can run binaries from before people were inhabiting the ISS and/or right around the time AOL was big enough to purchase Time Warner.
  - johnea 3y ago
    Thank you! I was going to point this out...
SanderNL 3y ago
They don’t need to migrate, but it’s usually a good idea to keep up to date. It’s indeed somewhat of a thing if you have a lot of systems and you have uptime requirements. Plan accordingly and it’ll be fine.
joelthelion 3y ago
How is docker any different? Don't docker containers share the kernel with the host?
- zajio1am 3y ago
  Linux have stable kernel ABI, while BSD changes kernel ABI between major versions.

inferiorhuman 3y ago

Kernels are generally backwards compatible and major versions are explicitly supported for five (5) years. Over in Linux land you'll see builds for the different libcs and it seems to work just fine.

johncolanduoni 3y ago
The difference is that the container libc version and kernel version don't have to correspond to each other in any way on Linux (unless your kernel is like 20 years older, in which case it wouldn't support Docker anyway).
- inferiorhuman 3y ago
  The end result is that you're matching the executable to the libc variant instead of to the kernel. In practice it's not much of a difference, if at all. If you need to use older binaries, install the compatibility shims (or use a base jail based on the older version of FreeBSD). In terms of a support matrix there are two major versions of FreeBSD that are given active support, which is less than over in Linux land. All in all of all the concerns I'd have about using jails in a production environment, dealing with different major versions is not on that list.
  - monocasa 3y ago
    But the half the point of the container is per application libcs/userlands, so the kernel ABI is the only required constant.
    - inferiorhuman 3y ago
      In FreeBSD the userland and kernel move in unison, and with the backwards compatibility there's nothing stopping you from running a jail based on a previous version of FreeBSD. The compatX packages are only there to provide the older libraries if you want to run an older application on a newer jail/install. As was pointed out, you can run binaries from a FreeBSD version (4) that's probably older than most posters here. Still not seeing what the big fuss is about.
      - monocasa 3y ago
        the hangup is because half the point is to be able to take a random userland and run it on your current system without changing it. As soon as your userland bitrots because the kernel ABI changed, you've lost a lot of the benefit. And to be clear, you can not take a complete userland from 4.x. You have to reinstall your application on a newer userland with compat shims when you change the kernel. Ironically enough, the Linux syscall compat layer in FreeBSD may give you more stability here.
        inferiorhuman 3y ago
        You have to reinstall your application on a newer userland with compat shims when you change the kernel. No, you don't. You can absolutely take a jail based on a previous version of FreeBSD and run it on a newer version. I've not tried going as far back as 4.x, but I've done jail images of 9 and 10 on 11 and 12 hosts as part of a CI system. The shims you're talking about are not shims at all, just packages of the older libraries that a program built for an older version of FreeBSD would expect. You can go ahead and look at the package manifests if you don't believe me. They're literally just a handful of older shared libraries. Honestly, I can't remember a time when the syscall table changed in a backwards incompatible manner. The compat packages are only needed if you're trying to run an older program with a newer userland. If you're running an older jail in a newer freebsd host, you can run your program as-is because those libraries have already been installed.
        3y ago
- bandrami 3y ago
  I thought that too, until the syscall semantics changed between RHEL 6 and RHEL 7. Spent way more time debugging that than I wanted to.

throw0101a 3y ago

There are compat libraries that are available, so if you are running 13.x (the most current major release) but want to run binaries originally compiled under 12.x, you install: * https://www.freshports.org/misc/compat12x/ https://www.freshports.org/misc/compat12x/ See further: * https://www.freshports.org/misc/compat11x/ https://www.freshports.org/misc/compat11x/ * https://www.freshports.org/misc/compat10x/ https://www.freshports.org/misc/compat10x/ […] * https://www.freshports.org/misc/compat4x/ https://www.freshports.org/misc/compat4x/

nucleardog 3y ago
And for anyone that isn't intimately familiar with FreeBSD's release history or anything, I feel like it's worth pointing out that the 4.x major release dates from over 23 years ago. You can run binaries from before people were inhabiting the ISS and/or right around the time AOL was big enough to purchase Time Warner.
- johnea 3y ago
  Thank you! I was going to point this out...

SanderNL 3y ago

They don’t need to migrate, but it’s usually a good idea to keep up to date. It’s indeed somewhat of a thing if you have a lot of systems and you have uptime requirements. Plan accordingly and it’ll be fine.

joelthelion 3y ago

How is docker any different? Don't docker containers share the kernel with the host?

zajio1am 3y ago
Linux have stable kernel ABI, while BSD changes kernel ABI between major versions.