4 ms·

I suspect interest in FreeBSD will rise with the demise of CentOS. What are the DevOps best practices for working with and deploying to jails instead of Docker

by mikece 6y ago

I suspect interest in FreeBSD will rise with the demise of CentOS. What are the DevOps best practices for working with and deploying to jails instead of Docker?

ggm 6y ago
Absent a good equivalent of Dockerfile, and compose features, I think this is doomed. (Longterm freebsd user btw)
- hda111 6y ago
  What’s wrong with BastilleBSD?
  - inetknght 6y ago
    I imagine the only thing wrong with it is that it's not nearly as well known as Docker.
    - ggm 6y ago
      Yes, mostly this. I think its targeting a different deployment model.
      - tpetry 6y ago
        I am not using it but the features sound fantastic, like a complete network for every jail without complex docker network interface solutions. Why dont they do a freebsd container runtime based on the jail semantics? They sound much more effective than the linux ones. Ok, the problem would be on how to run linux executables on freebsd.
        ggm 6y ago
        Ok, the problem would be on how to run linux executables on freebsd. That problem is mostly solved. Has been for years except for marginal sysctl() calls unlikely in datacenter usage of Linux.
  - ggm 6y ago
    Nothing, except it doesn't look to be doing additive composition on top of images and it doesn't look familiar if you were targeting pods in a k8s style deployment. As a thing in itself? Its fine. If you have 5+ years investment in docker, the missing, parts here are going to hurt. Mainly, what's missing is helpful evidence of transition. E.g. I have a docker image i got from git which works as a one-shot execution chain to run xml2rfc and I got a version for cramdown which worked mostly the same way to do rfc/drafts from markdown. Where's the BastilleBSD equivalent? How do I translate a Dockerfile to a Bastille script?
tofaz 6y ago
There is a great simple framework that can ease a lot the deployment and management of Jails called BastilleBSD. You can install it with pkg install bastille. https://bastillebsd.org/ https://bastillebsd.org/
UI_at_80x24 6y ago
They are vastly different creatures as you know, but in my experience on long-lived-but-must-keep-running services, fat-jails are a MUST. OS package mismatches will bite you in the ass years down the road if you try to maintain thin-jails. I manage alot of sites that Management refuses to invest any dev-time to upgrade old Python2.7 infrastructure. Having fat-Jails allows me to keep everything else upgraded. pf is a fantastic firewall, but the syntax in OpenBSD is different then in FreeBSD. VNET is worth it's weight in gold. Each jails has it's own network stack. That let's you avoid funky localhost workarounds. iocage is a great framework for jails management (but it isn't necessary) you can do everything from jail.conf. Personally I prefer it for it's ZFS integration.
- rsync 6y ago
  Hmm ... I was about to respond and say the opposite :) My (personal) unbound jail has 95 files in it. The reason it has so many files is because it is simultaneously my 'nsd' jail. Which is to say, I actually point two different 'jail' commands at the same directory - each with a different IP and command line: /usr/sbin/jail /jails/dns ns1 1.2.3.4 /nsd/nsd -c /nsd/nsd.conf /usr/sbin/jail /jails/dns dns 1.2.3.5 /unbound/unbound -c /unbound/unbound.conf So /jails/dns is a holder for two different dns servers and they run, jailed, from the same root. It's a little unorthodox but in this case I find it to be elegant.
  - toast0 6y ago
    I agree. I would run jails with very few files. A static executable, a config file, a log file, and some pipes maybe. Don't need anything else, don't want anything else (log file could be avoided with syslog, perhaps)
    - rsync 6y ago
      Bonus points: chflag schg the exe and config files such that they cannot be altered ... or better yet, make it a zfs filesystem and mount it read-only ...
  - UI_at_80x24 6y ago
cesarb 6y ago
> I suspect interest in FreeBSD will rise with the demise of CentOS. Indeed, the first thing I thought when seeing this post was "I wonder how long the support for each FreeBSD release is". Looking at Wikipedia (https://en.wikipedia.org/wiki/FreeBSD#Version_history https://en.wikipedia.org/wiki/FreeBSD#Version_history), it seems to be around 5 years. Still no match for the 10 year support CentOS users expected; IMO, 5 years is too short when installing out of cycle (for instance, since the current stable 12.x was released two years ago, if I installed it today on a new computer I'd have only 3 years of support left).
- thijsvandien 6y ago
  This is much less of an issue with FreeBSD because the changes between releases are relatively small. Some systems out there had 20 years worth of upgrades, from v4 all the way to v12. I wouldn't like to try that on Linux.
  - emidln 6y ago
    You could probably get pretty close with Red Hat Linux 7.2 or 7.3 to 8 to 9 to Fedora Core 1 and on. There might be a break somewhere that the upgrade doesn't work, but nothing sticks out in my mind. Anaconda is a seriously impressive install/upgrade system.
- gspr 6y ago
  > 5 years If indeed 5 years suffices, Debian LTS may also be a candidate.

by mikece 6y ago

I suspect interest in FreeBSD will rise with the demise of CentOS. What are the DevOps best practices for working with and deploying to jails instead of Docker?

ggm 6y ago

Absent a good equivalent of Dockerfile, and compose features, I think this is doomed. (Longterm freebsd user btw)

hda111 6y ago
What’s wrong with BastilleBSD?
- inetknght 6y ago
  I imagine the only thing wrong with it is that it's not nearly as well known as Docker.
  - ggm 6y ago
    Yes, mostly this. I think its targeting a different deployment model.
    - tpetry 6y ago
      I am not using it but the features sound fantastic, like a complete network for every jail without complex docker network interface solutions. Why dont they do a freebsd container runtime based on the jail semantics? They sound much more effective than the linux ones. Ok, the problem would be on how to run linux executables on freebsd.
      - ggm 6y ago
        Ok, the problem would be on how to run linux executables on freebsd. That problem is mostly solved. Has been for years except for marginal sysctl() calls unlikely in datacenter usage of Linux.
- ggm 6y ago
  Nothing, except it doesn't look to be doing additive composition on top of images and it doesn't look familiar if you were targeting pods in a k8s style deployment. As a thing in itself? Its fine. If you have 5+ years investment in docker, the missing, parts here are going to hurt. Mainly, what's missing is helpful evidence of transition. E.g. I have a docker image i got from git which works as a one-shot execution chain to run xml2rfc and I got a version for cramdown which worked mostly the same way to do rfc/drafts from markdown. Where's the BastilleBSD equivalent? How do I translate a Dockerfile to a Bastille script?

tofaz 6y ago

There is a great simple framework that can ease a lot the deployment and management of Jails called BastilleBSD. You can install it with pkg install bastille. https://bastillebsd.org/ https://bastillebsd.org/

UI_at_80x24 6y ago

They are vastly different creatures as you know, but in my experience on long-lived-but-must-keep-running services, fat-jails are a MUST. OS package mismatches will bite you in the ass years down the road if you try to maintain thin-jails. I manage alot of sites that Management refuses to invest any dev-time to upgrade old Python2.7 infrastructure. Having fat-Jails allows me to keep everything else upgraded. pf is a fantastic firewall, but the syntax in OpenBSD is different then in FreeBSD. VNET is worth it's weight in gold. Each jails has it's own network stack. That let's you avoid funky localhost workarounds. iocage is a great framework for jails management (but it isn't necessary) you can do everything from jail.conf. Personally I prefer it for it's ZFS integration.

rsync 6y ago
Hmm ... I was about to respond and say the opposite :) My (personal) unbound jail has 95 files in it. The reason it has so many files is because it is simultaneously my 'nsd' jail. Which is to say, I actually point two different 'jail' commands at the same directory - each with a different IP and command line: /usr/sbin/jail /jails/dns ns1 1.2.3.4 /nsd/nsd -c /nsd/nsd.conf /usr/sbin/jail /jails/dns dns 1.2.3.5 /unbound/unbound -c /unbound/unbound.conf So /jails/dns is a holder for two different dns servers and they run, jailed, from the same root. It's a little unorthodox but in this case I find it to be elegant.
- toast0 6y ago
  I agree. I would run jails with very few files. A static executable, a config file, a log file, and some pipes maybe. Don't need anything else, don't want anything else (log file could be avoided with syslog, perhaps)
  - rsync 6y ago
    Bonus points: chflag schg the exe and config files such that they cannot be altered ... or better yet, make it a zfs filesystem and mount it read-only ...
- UI_at_80x24 6y ago

cesarb 6y ago

> I suspect interest in FreeBSD will rise with the demise of CentOS. Indeed, the first thing I thought when seeing this post was "I wonder how long the support for each FreeBSD release is". Looking at Wikipedia (https://en.wikipedia.org/wiki/FreeBSD#Version_history https://en.wikipedia.org/wiki/FreeBSD#Version_history), it seems to be around 5 years. Still no match for the 10 year support CentOS users expected; IMO, 5 years is too short when installing out of cycle (for instance, since the current stable 12.x was released two years ago, if I installed it today on a new computer I'd have only 3 years of support left).

thijsvandien 6y ago
This is much less of an issue with FreeBSD because the changes between releases are relatively small. Some systems out there had 20 years worth of upgrades, from v4 all the way to v12. I wouldn't like to try that on Linux.
- emidln 6y ago
  You could probably get pretty close with Red Hat Linux 7.2 or 7.3 to 8 to 9 to Fedora Core 1 and on. There might be a break somewhere that the upgrade doesn't work, but nothing sticks out in my mind. Anaconda is a seriously impressive install/upgrade system.
gspr 6y ago
> 5 years If indeed 5 years suffices, Debian LTS may also be a candidate.