6 ms·

Very nice I used jails on FreeBSD and nothing in Linux comes close. Yes, it is not a pointy-clicky setup like Linux likes to do. But IMHO Jails are far more

by jmclnx 3y ago

Very nice I used jails on FreeBSD and nothing in Linux comes close. Yes, it is not a pointy-clicky setup like Linux likes to do. But IMHO Jails are far more secure, in a way: you get what you 'pay' for.

betaby 3y ago
Please provide technical details of `Jails are far more secure`.
- throw0101c 3y ago
  Multiple CVEs for escaping? E.g.: * https://blog.aquasec.com/cve-2022-0185-linux-kernel-container-escape-in-kubernetes https://blog.aquasec.com/cve-2022-0185-linux-kernel-containe... * https://securitylabs.datadoghq.com/articles/dirty-pipe-container-escape-poc/ https://securitylabs.datadoghq.com/articles/dirty-pipe-conta... * https://snyk.io/learn/docker-security/top-5-vulnerabilities/ https://snyk.io/learn/docker-security/top-5-vulnerabilities/ * https://www.container-security.site/attackers/container_breakout_vulnerabilities.html https://www.container-security.site/attackers/container_brea... The only escaping of jails that I've heard of in the last ~20 years is one not in the jails code, but tunnelling out through devfs: * https://www.freebsd.org/security/advisories/FreeBSD-SA-14:07.devfs.asc https://www.freebsd.org/security/advisories/FreeBSD-SA-14:07...
  - betaby 3y ago
    No? Because many of them Docker specific. Neither I mentioned Docker anywhere, not Docker is a good example of software with a good security record. Another link lumps together k8s, runc and so on. I specifically mentioned systemd-nspaw in the beginning as contender to jails, not other solutions.
  - tptacek 3y ago
    Misleading: the most important vulnerabilities that break shared-kernel isolation systems like Jails, Zones, and containers are kernel LPEs, and they aren’t reported as “escapes”.
gtirloni 3y ago
What makes Jails so far ahead of containers in your opinion?
- doublerabbit 3y ago
  Not op, but for me: You can setup Linux in a jail. You can nest bhyve virtual machines in a jail. You can assign individual NICs to a jail. You can encrypt with ZFS a jail. You can run browsers in a jail with full set of features.
  - betaby 3y ago
    All of those available on Linux and even more, see https://www.man7.org/linux/man-pages/man1/systemd-nspawn.1.html https://www.man7.org/linux/man-pages/man1/systemd-nspawn.1.h...
    - doublerabbit 3y ago
      I don't see the guide to run FreeBSD in a container from that man page. And what about if my Linux Distro doesn't use system-md, like Slackware.
      - betaby 3y ago
        > And what about if my Linux Distro doesn't use system-md, like Slackware. It's a choice of Slackware not use systemd infra. However on Linuxes you have more that one choice to run containers, your next option would be LXC/LDX https://docs.slackware.com/howtos:misc:lxc https://docs.slackware.com/howtos:misc:lxc
        doublerabbit 3y ago
        Sure and my choice not to use it too. Well aware of LXC too. I'm not claiming Linux can't. I was stating why. I know Linux can do what FreeBSD can do, but BSD does it better. However both are pretty much mimicked concept from Solaris Zones
irusensei 3y ago
I don’t know about the low level stuff but LXC is pretty much the same end result as FreeBSD jails. At least for my very basic needs that is.
johnisgood 3y ago
What about `firejail` on Linux? Is it not much more secure than FreeBSD Jails which is just based on chroot?
- bell-cot 3y ago
  FreeBSD Jails are conceptually just based on chroot. Their actual design and implementation are a few orders of magnitude more secure.
  - mardifoufs 3y ago
    A few order of magnitudes? Can you be more specific? I'm genuinely wondering what you could be referring to that I've missed.
    - tptacek 3y ago
      chroot more or less isn't secure at all. All they're really saying is that Jails aren't built on top of chroot, but rather are a ground-up reimplementation motivated by improving chroot.
      - mardifoufs 3y ago
        Ah! For some reason I thought jails were built on something similar to chroot at the kernel level, meaning that it also came with the same security caveats. (to be more accurate, I knew that it was not "on top" in the same way as Linux containers are built on top of chroot. )
    - bell-cot 3y ago
      https://en.wikipedia.org/wiki/Operating_system-level_virtualization#Implementations https://en.wikipedia.org/wiki/Operating_system-level_virtual... or https://en.wikipedia.org/wiki/Freebsd_jail https://en.wikipedia.org/wiki/Freebsd_jail
      - NexRebular 3y ago
tptacek 3y ago
Jails are not, in fact, far more secure than Linux containers. Arguably they're a better, more coherent design, but both are shared-kernel isolation schemes, and the high-order bit of their security is exactly the same. The best argument you'll come up with is that FreeBSD kernel LPEs are rarer than those of Linux, but now we're just back to the standard Linux vs. FreeBSD security debate, which never goes anywhere.
- zie 3y ago
  I agree with all of this, with perhaps one tiny addendum. Defaults matter, and last I checked Docker/container defaults are miserable for security. I'm guessing if you randomly inspected production container vs jails out in the world, way more container solutions would have giant holes in them compared to jails.
  - cpuguy83 3y ago
    Defaults on Docker specifically are pretty locked down. Even if if the container process has the root uid it is not able to do most things normal root can do, and is pretty much limited to being able to access root owned files and binding to privileged ports (which in a container doesn't tend to matter).
    - yjftsjthsd-h 3y ago
      > Defaults on Docker specifically are pretty locked down. That is strongly dependent on your threat model. The default docker configuration completely bypasses the firewall, making it trivial for containers to be exposed to the open internet with no way for admins to prevent it[0]. Likewise, I hesitate to call docker's default of running as root safe since it means anyone with access to the docker socket immediately has root on the host. [0] It is quite easy for someone even slightly inexperienced to accidentally write, say `-p 1234:1234` instead of `-p 127.0.0.1:1234:1234` and thereby cause a security incident or near-miss; ask me how I know.
      - cpuguy83 3y ago

by jmclnx 3y ago

betaby 3y ago

Please provide technical details of `Jails are far more secure`.

throw0101c 3y ago
Multiple CVEs for escaping? E.g.: * https://blog.aquasec.com/cve-2022-0185-linux-kernel-container-escape-in-kubernetes https://blog.aquasec.com/cve-2022-0185-linux-kernel-containe... * https://securitylabs.datadoghq.com/articles/dirty-pipe-container-escape-poc/ https://securitylabs.datadoghq.com/articles/dirty-pipe-conta... * https://snyk.io/learn/docker-security/top-5-vulnerabilities/ https://snyk.io/learn/docker-security/top-5-vulnerabilities/ * https://www.container-security.site/attackers/container_breakout_vulnerabilities.html https://www.container-security.site/attackers/container_brea... The only escaping of jails that I've heard of in the last ~20 years is one not in the jails code, but tunnelling out through devfs: * https://www.freebsd.org/security/advisories/FreeBSD-SA-14:07.devfs.asc https://www.freebsd.org/security/advisories/FreeBSD-SA-14:07...
- betaby 3y ago
  No? Because many of them Docker specific. Neither I mentioned Docker anywhere, not Docker is a good example of software with a good security record. Another link lumps together k8s, runc and so on. I specifically mentioned systemd-nspaw in the beginning as contender to jails, not other solutions.
- tptacek 3y ago
  Misleading: the most important vulnerabilities that break shared-kernel isolation systems like Jails, Zones, and containers are kernel LPEs, and they aren’t reported as “escapes”.

gtirloni 3y ago

What makes Jails so far ahead of containers in your opinion?

doublerabbit 3y ago
Not op, but for me: You can setup Linux in a jail. You can nest bhyve virtual machines in a jail. You can assign individual NICs to a jail. You can encrypt with ZFS a jail. You can run browsers in a jail with full set of features.
- betaby 3y ago
  All of those available on Linux and even more, see https://www.man7.org/linux/man-pages/man1/systemd-nspawn.1.html https://www.man7.org/linux/man-pages/man1/systemd-nspawn.1.h...
  - doublerabbit 3y ago
    I don't see the guide to run FreeBSD in a container from that man page. And what about if my Linux Distro doesn't use system-md, like Slackware.
    - betaby 3y ago
      > And what about if my Linux Distro doesn't use system-md, like Slackware. It's a choice of Slackware not use systemd infra. However on Linuxes you have more that one choice to run containers, your next option would be LXC/LDX https://docs.slackware.com/howtos:misc:lxc https://docs.slackware.com/howtos:misc:lxc
      - doublerabbit 3y ago
        Sure and my choice not to use it too. Well aware of LXC too. I'm not claiming Linux can't. I was stating why. I know Linux can do what FreeBSD can do, but BSD does it better. However both are pretty much mimicked concept from Solaris Zones

irusensei 3y ago

I don’t know about the low level stuff but LXC is pretty much the same end result as FreeBSD jails. At least for my very basic needs that is.

johnisgood 3y ago

What about `firejail` on Linux? Is it not much more secure than FreeBSD Jails which is just based on chroot?

bell-cot 3y ago
FreeBSD Jails are conceptually just based on chroot. Their actual design and implementation are a few orders of magnitude more secure.
- mardifoufs 3y ago
  A few order of magnitudes? Can you be more specific? I'm genuinely wondering what you could be referring to that I've missed.
  - tptacek 3y ago
    chroot more or less isn't secure at all. All they're really saying is that Jails aren't built on top of chroot, but rather are a ground-up reimplementation motivated by improving chroot.
    - mardifoufs 3y ago
      Ah! For some reason I thought jails were built on something similar to chroot at the kernel level, meaning that it also came with the same security caveats. (to be more accurate, I knew that it was not "on top" in the same way as Linux containers are built on top of chroot. )
  - bell-cot 3y ago
    https://en.wikipedia.org/wiki/Operating_system-level_virtualization#Implementations https://en.wikipedia.org/wiki/Operating_system-level_virtual... or https://en.wikipedia.org/wiki/Freebsd_jail https://en.wikipedia.org/wiki/Freebsd_jail
    - NexRebular 3y ago

tptacek 3y ago

Jails are not, in fact, far more secure than Linux containers. Arguably they're a better, more coherent design, but both are shared-kernel isolation schemes, and the high-order bit of their security is exactly the same. The best argument you'll come up with is that FreeBSD kernel LPEs are rarer than those of Linux, but now we're just back to the standard Linux vs. FreeBSD security debate, which never goes anywhere.

zie 3y ago
I agree with all of this, with perhaps one tiny addendum. Defaults matter, and last I checked Docker/container defaults are miserable for security. I'm guessing if you randomly inspected production container vs jails out in the world, way more container solutions would have giant holes in them compared to jails.
- cpuguy83 3y ago
  Defaults on Docker specifically are pretty locked down. Even if if the container process has the root uid it is not able to do most things normal root can do, and is pretty much limited to being able to access root owned files and binding to privileged ports (which in a container doesn't tend to matter).
  - yjftsjthsd-h 3y ago
    > Defaults on Docker specifically are pretty locked down. That is strongly dependent on your threat model. The default docker configuration completely bypasses the firewall, making it trivial for containers to be exposed to the open internet with no way for admins to prevent it[0]. Likewise, I hesitate to call docker's default of running as root safe since it means anyone with access to the docker socket immediately has root on the host. [0] It is quite easy for someone even slightly inexperienced to accidentally write, say `-p 1234:1234` instead of `-p 127.0.0.1:1234:1234` and thereby cause a security incident or near-miss; ask me how I know.
    - cpuguy83 3y ago