9 ms·

Back to FreeBSD: Part 1

222 points by enz 8mo ago

nesarkvechnep 8mo ago
I’m always going to like articles introducing people to FreeBSD.
palata 8mo ago
Nice article! > To solve the distribution and isolation problem, Linux engineers built a set of kernel primitives (namespaces, cgroups, seccomp) and then, in a very Linux fashion, built an entire ecosystem of abstractions on top to “simplify” things: [...] Somehow we ended up with an overengineered mess of leaky abstractions Not sure I like the value judgement here. I think it's more of a consequence of Linux' success. I am convinced that if it was reversed (Linux was niche and *BSD the norm), then a ton of abstractions would come, and the average user would "use an overengineered mess" because they don't know better (or don't care or don't have a need to care). Not that I like it when people ship their binary in a 6G docker image. But I don't think it's fair to put that on "those Linux engineers".
- realusername 8mo ago
  I don't think it's necessarily true, compare the BSD utils to the GNU utils and the style difference is very visible. On the other hand, I don't think the comparison between jails and docker is fair. What made Docker popular is the reusability of the containers, certainty not the sandboxing which in the early days was very leaky.
  - NooneAtAll3 8mo ago
    what do you mean by reusability?
    - deleted 8mo ago
      [deleted]
    - maxloh 8mo ago
      For example, you can build a Python image, and reuse it on every Python apps you have.
      - fragmede 8mo ago
        And for the whole world, too. I don't need to build my own local stripped down version of Alpine Linux with python, somebody's already dike that for me.
        irusensei 8mo ago
        I don't like that aspect of OCI containers. You shouldn't be running or building on top of random images made by unknowns.
        JCattheATM 8mo ago
        Convenience beats security every time.
        cindyllm 8mo ago
        [dead]
  - shevy-java 8mo ago
    > compare the BSD utils to the GNU utils and the style difference is very visible. Well, what style difference exactly? GNU utils tend to be more verbose. Other than that, what is the difference in style?
    - adrian_b 8mo ago
      I do not know which is the difference, but you really feel a difference. It might be of homogeneity, i.e. the FreeBSD tools behave in a consistent way, while there are significant differences between the Linux tools, depending on which were the opinions of their particular authors about how the traditional UNIX tools should be changed. For instance, at some point in time, long ago, in Linux the traditional "ifconfig" and a few related commands have been replaced by "ip", for managing networking. The Linux "ifconfig" needed an upgrade, as it could do only a small fraction of what the FreeBSD "ifconfig" could do. Nevertheless, until today, decades later, I have been unable to stop hating the Linux "ip". I cannot say why, because in other cases when some command-line or GUI utility that I had used for many years was replaced by an alternative I instantly recognized that the new UI was better and I never wanted to use the old UI again. So while both FreeBSD and Linux have started with the same traditional UNIX utilities, they have evolved divergently and now they frequently feel quite differently, in the sense that the various options in commands or in configuration files may match your expectations only when taking into account the identity of the OS. Overall FreeBSD has been more conservative, but there are also cases when it has made bigger changes, but such changes seem more carefully planned and less haphazard than in the Linux world.
    - Lammy 8mo ago
      They're more consistent with each other, like how they all respond to SIGINFO when you Ctrl+T.
  - bubblewand 8mo ago
    Indeed, that Docker is functionally a cross-distro rolling release package manager, configuration standard, and service supervisor[1] is the appeal to me. Any isolation it achieves is necessary for that to all work reliably, but is not why I use it. Inability to find a service I want to run on Github and 95+% of the time to be able to configure it and have it running and fully managed with usually just a one-liner shell script like 10 minutes later just by finding an existing docker image is the thing I’d lose with jails. That’s all of the value of docker to me personally. Jails could be a building block toward that, but last I checked there’s no deep and up-to-date library of “packages” I can reach for, using jails, which makes it pretty much useless to me. 1: I have like eight or nine services running on my home Debian system, they all auto-restart and come back up on reboot, and I’ve not had to touch Systemd once on that machine.
- jacquesm 8mo ago
  I don't agree with that. FreeBSD has more of an engineering than a hacking mentality and it shows in the various architectural choices. And containers really are a VM-light, so you might as well use the real thing, in fact, VMWare for a long time thought that their images would be a container like thing and many larger installations used them as such.
lifeisstillgood 8mo ago
I ran a whole company on top of FreeBSD back in the day (2005 ish). It was great, and ran all my personal pcs the same way (hell, refusing to install windows to try out this bitcoin idea is even now a good idea). But somehow Linux still took over my personal and professional life. Going back seems nice but there need to be a compelling reason -docker is fine, the costs don’t add up any more. I do t have a real logical argument beyond that.
- dijit 8mo ago
  Yeah, I have a similar situation; FreeBSD is a great operating system, but the sheer amount of investment in Linux makes all the warts semi-tolerable. I'm sure some people have a sunk-cost feeling with Linux and will get defensive of this, but ironically this was exactly the argument I had heard 20 years ago - and I was defensive about it myself then.. This has only become more true though. It's really hard to argue against Linux when even architecturally poor decisions are papered over by sheer force of will and investment; so in a day-to-day context Linux is often the happy path even though the UX of FreeBSD is more consistent over time.
- flipped 8mo ago
  Never understood why satoshi was a prime windows user.
  - earthscienceman 8mo ago
    I know this comment is effectively a side tangent on a side tangent. but that was always the strangest thing to me as well. I remember in 2012 when I was debating fiddling around with Bitcoin. that was one of the things that turned me off. I was sure that there was no way something as brilliant as this was supposed to be was developed by windows user. Which surely says something about all these ideological purity tests
    - dijit 8mo ago
      Windows developers (like sysadmins) are of two kinds in my experience. People who don't understand shit about how the system behaves and are comfortable with that. "I install a package, I hit the button, it works" .. and People who understand very deeply how computers work, and genuinely enjoy features of the NT Kernel, like IOCP and the performance counters they offer to userland. What's weird to me is that the competence is bimodal; you're either in the first camp or the second. With Linux (+BSD/Solaris etc;) it's a lot more of a spectrum. I've never understood exactly why this is, but it's consistent. There's no "middle-good" Windows developer.
matheus-rr 8mo ago
The jails vs containers framing is interesting but I think it misses why Docker actually won. It wasn't the isolation tech. It was the ecosystem: Dockerfiles as executable documentation, a public registry, and compose for local dev. You could pull an image and have something running in 30 seconds without understanding anything about cgroups or namespaces. FreeBSD jails were technically solid years before Docker existed, but the onboarding story was rough. You needed to understand the FreeBSD base system first. Docker let you skip all of that. That said, I've been seeing more people question the container stack complexity recently. Especially for smaller deployments where a jail or even a plain VM with good config management would be simpler and more debuggable. The pendulum might be swinging back a bit for certain use cases.
- sthuck 8mo ago
  I don't think article misses it, it's exactly the point it makes
- torstenvl 8mo ago
  > Jails solve the isolation problem beautifully, but they don't have a native answer to shipping. That gap is real, and it's one of the main reasons the ecosystem around jails feels underdeveloped compared to Docker's world. The link literally uses the term ecosystem. Several times actually.
- steve1977 8mo ago
  > You could pull an image and have something running in 30 seconds without understanding anything Fixed that for you ;)
- steve1977 8mo ago
  Maybe FreeBSD doesn't want a jails "ecosystem"?
- wolvoleo 8mo ago
  Jails were never going to 'win' because they're only on an OS with 0.1% marketshare. But it's not a competition. FreeBSD does its thing and Linux does another. That's why I use FreeBSD.
NooneAtAll3 8mo ago
"failed to verify your browser"
- m132 8mo ago
  Getting the same thing, "Failed to verify your browser. Code 11". Some noise about WebGL in the browser console, getExtension() invoked on a null reference. LibreWolf on Linux + resist fingerprinting. Maybe opting for a better-written WAF could boost the reach?
flipped 8mo ago
Is there any technical writeup which explains how the isolation exactly works, on containers and VMs? I have always heard the high level arguments of weak isolation, same kernel, etc but never the implementation details.
flipped 8mo ago
Anyone looking to use jails might find BastileBSD helpful. It's a nice and modern jail manager.
- paul_h 8mo ago
  I was looking at TrueNAS CORE to see if it was a viable way to bsd-jail Linux containers. I'm really only doing this to get some protection from supply chain attacks given I'm fairly promiscuous at git-clone-and-run-a-build. Before that I was aiming for the same with Bastille and had got to the give up stage because it felt too fiddly to set up. This was a year ago. Maybe its better now
  - Brian_K_White 8mo ago
    Unfortunately trunas core is dead now. zVault is a fork that is effortless to migrate in-place, but pointless because it has had no updates since the fork, it's no different from just continuing to run the derelict final version of truenas core. That just leaves xigmanas which I have not tried yet, but looks like a simpler more pure nas without the jails or vm manager, which people have told me can be filled by bastille. Or really, I'm thinking rather than even xigmanas it probably makes more sense to just use plain freebsd and never get stuck like this again. The host is stuck at 13.3. 13.3 went fully EOL December 2024. The pkg repos don't even supply packages for that any more. I have a bunch of services that run in jails, and currently I can just barely squeak by by "illegally" updating the jails to 13.5. It's not officially supported by upstream freebsd but I seem to be getting away with it for now. But even 13.5 is not going to last much longer. Then what? So really the FreeNAS ui was nice an all, but not so nice as to be worth being stuck like this now. I probably should have just skipped it and just used plain freebsd which would never have had any such problem. So maybe assuming zvault continues to not update when I finally need to move some jail past 13.5, maybe the next move is not even to xigmanas but just plain freebsd.
user3939382 8mo ago
I switched my startup’s whole infra to FreeBSD a couple months ago. Found a use after free bug that Linux’s memory management was just fine with in Gnome XSLT lib that FreeBSD properly refused. Other than that smooth sailing, jails work great. After IBM destroyed CentOS, all the Xorg politics nonsense, the list goes on with Linux, not interested. I just want something quiet and boring and stable and correctly designed. NetBSD would be my first choice but they don’t get the $ they need for drivers.
- manuelabeledo 8mo ago
  You don’t need to follow the news cycle to use an operating system.
  - user3939382 8mo ago
    I do follow the news cycle and if I’m hearing about a software package in it, something is wrong with the people making the software and I don’t trust them. Software is an engineering discussion or at least it’s supposed to be. Here’s my community guidelines: everyone be nice and respectful engage in good faith and focus on the math. Being social is fine so long that it doesn’t become a diversion from the engineering discussion. We’re talking about code not a philosophical treatise. There are civil ways to settle disagreements. I’m so sick to death of the politics.
    - manuelabeledo 8mo ago
      It sounds like software projects are built by humans. Nothing wrong with that. Unless we’re assuming here that the BSD community is free from that.
      - user3939382 8mo ago
        The engineers I’ve worked with my whole career had no problem getting the work done without getting into giant political debates. If your politics comes before the engineering I don’t want anything to do with it.
jmclnx 8mo ago
>but they don't have a native answer to shipping I am not quite sure what this means. I had a jail a few years ago and I remember there was a utility to "back" the jail up so you could put it on another system. Are there constraints with that utility. It seemed to work, maybe I am forgetting something ? In any case I still think Jails are much better than the things Linux has. To me, it is creating a jail that is more difficult. There were ports that made it easier, I used one of them, but that port was abandoned at some point. I think it was "ezjail".
efortis 8mo ago
A two-server networking setup with VNET Jails: https://ericfortis.com/blog/freebsd-jails-network-setup https://ericfortis.com/blog/freebsd-jails-network-setup
shevy-java 8mo ago
> FreeBSD is worth a brief aside here, because it differs from Linux in a fundamental way. Linux is a kernel. What most people call "Linux" is actually that kernel combined with a GNU userland, a package ecosystem, and a set of choices that vary from distro to distro — Ubuntu, Fedora, and Arch are all running the same kernel but are meaningfully different systems underneath. It is not incorrect but ... do people really care about that distinction? Because in most situations I know of, when people refer to Linux, they almost never refer to the linux kernel. They refer to the whole operating system stack, which is typically put down via a distribution. So, Fedora, Gentoo, Arch, and so forth, are all "kind of" Linux. Barely anyone refers to the linux kernel if you look at all the discussions on the world wide web. > FreeBSD ships as a complete, coherent OS The BSDs often promote that aka "Linux is chaos, we are coherent and consistent operating system following intelligent design". Well ... this is the rise of worse is better, repeated: https://dreamsongs.com/WorseIsBetter.html https://dreamsongs.com/WorseIsBetter.html It is a great analogy that works on so many levels. Broken down to Linux versus the BSDs, I think 500 out of 500 top supercomputers running Linux kind of show which philosophy is better. The one that works better. That does not mean the BSDs are useless, but I am getting tired of the promo used by the BSD as "we are order, Linux is chaos". I compare this more to Lego building blocks. With Linux there is a stronger focus on having building blocks available. You can build up things. You have projects such as LFS/BLFS (Linux from scratch). The BSDs do not have something comparable. Which operating system is the better tinker OS? Which community created git? (Ok ok that was Linus so not really a community per se, but it originated from Linux and perhaps that was not an accident either.) > FreeBSD pioneered the practical implementation of what we now call containers. Ok great. Many modern programming languages learned from older languages; many of these older languages are dead now. You need to keep on innovating. Why is BSD so dead set on the past? > FreeBSD reached that third stage in 2000. Linux wouldn't get there until 2008 with LXC. Dumdedum ... it kind of sounds as if the FreeBSD guys are sad that Linux went on to dominate. It reminds me of NetBSD aka "we work on every toaster in the world". Then suddenly on a mailing list many years ago "wait a moment ... Linux now works on more toasters than we do". The BSDs don't seem to understand how momentum can be dominating. > Technical superiority doesn't win ecosystem wars. Linux won through a combination of fast decisions, the viral GPL licence, and strong enterprise backing from Red Hat and IBM. Then Google, Facebook, and Amazon happened — hungry for datacenters, developing tools to manage growing infrastructure at scale. They set the direction for the entire industry. Ok that flat out is incorrect. First - GPL worked well for the linux kernel, that is true. But the ecosystem includes many BSD-licences programs too, on Linux. So that explanation fails already here. LLVM has Apache License 2.0 which I kind of feel is a mix between GPL and BSD (not quite true but this is how I remember it). Then the claim is Linux won because of Red Hat. I actually find Red Hat annoying and I am glad to not depend on it. Linux is way bigger than Red Hat. IBM? I don't see what IBM did for Linux really. So that explanation also does not work. Google, Facebook, and Amazon - well, they profited from Linux. They didn't really ENABLE Linux. They would not have used Linux if Linux would have been useless. So that part came afterwards. So none of those explanations really work well here. > Linux rapidly went from "the free OS for people who can't afford commercial licences" to "the only acceptable OS for servers". That is true but not for the claims made, e. g. "because of Google". The more important question is: why did the BSDs fail? > To solve the distribution and isolation problem, Linux engineers built a set of kernel primitives (namespaces, cgroups, seccomp) and then, in a very Linux fashion, built an entire ecosystem of abstractions on top to “simplify” things No, that is also incorrect. cgroups are also very different to seccomp and the latter is even maintained independently: https://github.com/seccomp/libseccomp/releases https://github.com/seccomp/libseccomp/releases > Somehow we ended up with an overengineered mess of leaky abstractions for cloud-based, vendor-locked infrastructure. Wait a moment - he cites Docker. That's owned by a private company. What does this have to do with Linux? If company xyz does something based on FreeBSD, we would then say company xyz is responsible for FreeBSD failing or not failing? How does that work? > And this complexity has quietly reshaped how the industry thinks about deploying software. Today, if you want to run an application in a larger system, the implicit assumption is that you containerise it with Docker and orchestrate it with Kubernetes. Personally I find all this abstraction crap. With all their failures, though, things such as docker kind of present a "download this one file, then it will work fine". And that is kind of true. I saw that in in-campus use for life science faculty clusters and what not. It simplifies things for the admin there. People give a similar rationale for systemd. Personally I don't think systemd should exist, but there are people who benefit from it - that simply is a factual statement. All in all this is a very strange point of view from FreeBSD folks. At the least the NetBSD folks back then on the mailing list acknowledged the situation and then tried to find alternative strategies and in some ways succeeded (although I am not sure whether NetBSD right now runs on more toasters than Linux does - anyone has updated statistics for that?).
razighter777 8mo ago
I frequently see freeBSD jails as a highlighted feature, lauding their simplicity and ease of use. While I do admire them, there are benefits to the container approach used commonly on linux. (and maybe soon freebsd will better support OCI). First it's important to clarify "containers" are not an abstraction in the linux kernel. Containers are really an illusion achieved by use of a combination of user/pid/networking namespaces, bind mounts, and process isolation primitives through a userspace application(s) (podman/docker + a container runtime). OCI container tooling is much easier to use, and follows the "cattle not pets" philosophy, and when you're deploying on multiple systems, and want easy updates, reproducibility, and mature tooling, you use OCI containers, not LXC or freebsd jails. FreeBSD jails can't hold a candle to the ease of use and developer experience OCI tooling offers. > To solve the distribution and isolation problem, Linux engineers built a set of kernel primitives (namespaces, cgroups, seccomp) and then, in a very Linux fashion, built an entire ecosystem of abstractions on top to “simplify” things. This was an intentional design decision, and not a bad one! cgroups, namespaces, and seccomp are used extensively outside of the container abstraction. (See flatpak, systemd resource slices, firejail). By not tieing process isolation to the container abstraction, we can let non-container applications benefit from them. We also get a wide breadth of container runtime choices.
- Melatonic 8mo ago
  Jails have been around a long time in comparison I still see FreeBSD as being great for things like networking devices and storage controllers. You can apply a lot of the "cattle vs pets" design one level above that using VMs and orchestration tools.
- znpy 8mo ago
  > lauding their simplicity and ease of use Spawning a linux container is much simpler and faster than spawning a freebsd jail. I don’t know why i keep hearing about jails being better, they clearly aren’t.
  - Gud 8mo ago
    Sorry what? It's a 5 line configuration file to create a FreeBSD jail.
mono442 8mo ago
> FreeBSD reached that third stage in 2000. Linux wouldn't get there until 2008 with LXC. OpenVZ and Linux vserver are older than LXC and were commonly used, though they required a patched kernel.
- roryirvine 8mo ago
  Yeah, both VServer and Virtuozzo were roughly contemporaneous with the initial Xen release (2001-ish?), and only a few months behind jails. But Virtuozzo was hampered by being non-free - OpenVZ wasn't open sourced until 2-3 years later, by which time the damage had been done (but, of course, Xen headed in the opposite direction at roughly the same time!) And Linux-VServer was held back by being focussed so directly at virtual hosting providers - it positioned itself against fcgi-suexec, fcgid, and php-fpm (and was much more unwieldy than any of them) rather than jails or VZ. Both were more or less ignored until the late 2000s, by which time LXC had taken a lot of mindshare - allowing the "FreeBSD was years ahead with jails" meme to take root.
deleted 8mo ago
[deleted]
lizknope 8mo ago
> Technical superiority doesn't win ecosystem wars. Linux won through a combination of fast decisions, the viral GPL licence, and strong enterprise backing from Red Hat and IBM. Then Google, Facebook, and Amazon happened — hungry for datacenters, developing tools to manage growing infrastructure at scale. They set the direction for the entire industry. In the mid 1990's the hardware driver support on Linux was much broader. Copy / paste of my comment from last year about FreeBSD I installed Linux in fall 1994. I looked at Free/NetBSD but when I went on some of the Usenet BSD forums they basically insulted me saying that my brand new $3,500 PC wasn't good enough. The main thing was this IDE interface that had a bug. Linux got a workaround within days or weeks. https://en.wikipedia.org/wiki/CMD640 https://en.wikipedia.org/wiki/CMD640 The BSD people told me that I should buy a SCSI card, SCSI hard drive, SCSI CD-ROM. I was a sophomore in college and I saved every penny to spend $2K on that PC and my parents paid the rest. I didn't have any money for that. The sound card was another issue. I remember software based "WinModems" but Linux had drivers for some of these. Same for software based "Win Printers" When I finally did graduate and had money for SCSI stuff I tried FreeBSD around 1998 and it just seemed like another Unix. I used Solaris, HP-UX, AIX, Ultrix, IRIX. FreeBSD was perfectly fine but it didn't do anything I needed that Linux didn't already do.
- JCattheATM 8mo ago
  > FreeBSD was perfectly fine but it didn't do anything I needed that Linux didn't already do. That's pretty much it. A lot of the people I see using a BSD these days do so because they always have and they prefer what they know, which is fine, or they just want to be contrarian. Realistically, aside from edge cases in hardware support, you can do anything you want on any modern *nix. There's not even as much of a difference between distros as people claim. All the "I want an OS that gets out of my way" and similar reasons apply to most modern well-maintained distros these days. It's more personality and familiarity than anything objective.
  - lizknope 8mo ago
    I went from Slackware in 1994 to Red Hat in 1998 to Fedora when they split into Fedora and RH Enterprise. Every 2 or 3 years I will install a different distro in a VM and see "Okay, now I see what it's about." But I have no interest in switching as long as Fedora does everything I need. I don't really understand the people that distro hop. I just assume they are really young and I have work to do and a family to take care of.
smitty1e 8mo ago
Is this fair? Linux is to *BSD as VHS was to Betamax.
- sidkshatriya 8mo ago
  At one time this was an interesting comparison... but now Linux has gotten so much development that even if FreeBSD was Betamax and Linux VHS (in the past)... I would say that Linux is now DVD ... and FreeBSD still remains betamax. Don't get me wrong, FreeBSD is simple, elegant, consistent and well manicured. It seems to have picked up some pace again. I'm rooting for it.
  - smitty1e 8mo ago
    Indeed, there is no shame in being a dirt-simple system that Just Works. Likely is has a good place at the low end. In enterprise mode, you want something like an AWS to hide the pain of those large-scale details that Linux is bringing.
epistasis 8mo ago
Long emotional rant ahead, you have been warned. The poorly thought out adoption of parts of systemd, and in particular systemd-oomd, are making me yearn for FreeBSD. For all my computing career, I'd use Unix-alike because they let me develop software by having an idea, write some code, let it run in the terminal, chain things together, and see where it failed, and iterate. Terminals let this happen much faster than clicky GUI software, and contain a log of what was happening in a terminal pane (or gnus screen or tmux pane, because usually this is happening on big servers and compute clusters rather than my terminal/laptop) I could launch a bunch of panes, have several lines of investigation going, and come back to it a day or week later when I had time because the terminal kept a log of everything that happened. And a couple years ago I started noticing that things I thought I had launched would start disappearing. At first I thought I had started accidentally mispressing a key and killing a tmux pane rather than disconnecting. But no! When I finally went back to a in-person Linux workstation and saw it happen to the entire terminal window, I knew something major had changed. It turns out that something called systemd-oomd was added to a bunch of distributions that now kills only entire cgroups of processes at a time, rather than a single process offender. So now if you want to run processes and isolate the kill zone of a process, you have to wrap every freaking subprocess in an entire systemd-run wrapper or docker wrapper. And systemd-run won't work from many contexts, such as inside a Jupyter kernel. Major breaking changes on fundamental system behavior are a huge problem these days. It's one thing to let the OS kill processes more when there's a memory issue, fine, great, go ahead. But why kill all the lightweight processes that could give feedback to a user?! And why force non-portable process launching semantics, that aren't even consistent across the entire system?!? So infuriating.
DaveCharlieLen 8mo ago
> the viral GPL licence this was the key part for Linux success, but also it was a network effect thing, if you want users for a unix they likely would use Linux, so give them a driver on Linux and redirect people. Then having the kernel binding be malleable led to more people just opensourcing the drivers if they had an actual product or just vendor the linux kernel otherwise.

222 points by enz 8mo ago

I’m always going to like articles introducing people to FreeBSD.

Nice article! > To solve the distribution and isolation problem, Linux engineers built a set of kernel primitives (namespaces, cgroups, seccomp) and then, in a very Linux fashion, built an entire ecosystem of abstractions on top to “simplify” things: [...] Somehow we ended up with an overengineered mess of leaky abstractions Not sure I like the value judgement here. I think it's more of a consequence of Linux' success. I am convinced that if it was reversed (Linux was niche and *BSD the norm), then a ton of abstractions would come, and the average user would "use an overengineered mess" because they don't know better (or don't care or don't have a need to care). Not that I like it when people ship their binary in a 6G docker image. But I don't think it's fair to put that on "those Linux engineers".

realusername 8mo ago
I don't think it's necessarily true, compare the BSD utils to the GNU utils and the style difference is very visible. On the other hand, I don't think the comparison between jails and docker is fair. What made Docker popular is the reusability of the containers, certainty not the sandboxing which in the early days was very leaky.
- NooneAtAll3 8mo ago
  what do you mean by reusability?
  - deleted 8mo ago
    [deleted]
  - maxloh 8mo ago
    For example, you can build a Python image, and reuse it on every Python apps you have.
    - fragmede 8mo ago
      And for the whole world, too. I don't need to build my own local stripped down version of Alpine Linux with python, somebody's already dike that for me.
      - irusensei 8mo ago
        I don't like that aspect of OCI containers. You shouldn't be running or building on top of random images made by unknowns.
        JCattheATM 8mo ago
        Convenience beats security every time.
        cindyllm 8mo ago
        [dead]
- shevy-java 8mo ago
  > compare the BSD utils to the GNU utils and the style difference is very visible. Well, what style difference exactly? GNU utils tend to be more verbose. Other than that, what is the difference in style?
  - adrian_b 8mo ago
    I do not know which is the difference, but you really feel a difference. It might be of homogeneity, i.e. the FreeBSD tools behave in a consistent way, while there are significant differences between the Linux tools, depending on which were the opinions of their particular authors about how the traditional UNIX tools should be changed. For instance, at some point in time, long ago, in Linux the traditional "ifconfig" and a few related commands have been replaced by "ip", for managing networking. The Linux "ifconfig" needed an upgrade, as it could do only a small fraction of what the FreeBSD "ifconfig" could do. Nevertheless, until today, decades later, I have been unable to stop hating the Linux "ip". I cannot say why, because in other cases when some command-line or GUI utility that I had used for many years was replaced by an alternative I instantly recognized that the new UI was better and I never wanted to use the old UI again. So while both FreeBSD and Linux have started with the same traditional UNIX utilities, they have evolved divergently and now they frequently feel quite differently, in the sense that the various options in commands or in configuration files may match your expectations only when taking into account the identity of the OS. Overall FreeBSD has been more conservative, but there are also cases when it has made bigger changes, but such changes seem more carefully planned and less haphazard than in the Linux world.
  - Lammy 8mo ago
    They're more consistent with each other, like how they all respond to SIGINFO when you Ctrl+T.
- bubblewand 8mo ago
  Indeed, that Docker is functionally a cross-distro rolling release package manager, configuration standard, and service supervisor[1] is the appeal to me. Any isolation it achieves is necessary for that to all work reliably, but is not why I use it. Inability to find a service I want to run on Github and 95+% of the time to be able to configure it and have it running and fully managed with usually just a one-liner shell script like 10 minutes later just by finding an existing docker image is the thing I’d lose with jails. That’s all of the value of docker to me personally. Jails could be a building block toward that, but last I checked there’s no deep and up-to-date library of “packages” I can reach for, using jails, which makes it pretty much useless to me. 1: I have like eight or nine services running on my home Debian system, they all auto-restart and come back up on reboot, and I’ve not had to touch Systemd once on that machine.
jacquesm 8mo ago
I don't agree with that. FreeBSD has more of an engineering than a hacking mentality and it shows in the various architectural choices. And containers really are a VM-light, so you might as well use the real thing, in fact, VMWare for a long time thought that their images would be a container like thing and many larger installations used them as such.

lifeisstillgood 8mo ago

I ran a whole company on top of FreeBSD back in the day (2005 ish). It was great, and ran all my personal pcs the same way (hell, refusing to install windows to try out this bitcoin idea is even now a good idea). But somehow Linux still took over my personal and professional life. Going back seems nice but there need to be a compelling reason -docker is fine, the costs don’t add up any more. I do t have a real logical argument beyond that.

dijit 8mo ago
Yeah, I have a similar situation; FreeBSD is a great operating system, but the sheer amount of investment in Linux makes all the warts semi-tolerable. I'm sure some people have a sunk-cost feeling with Linux and will get defensive of this, but ironically this was exactly the argument I had heard 20 years ago - and I was defensive about it myself then.. This has only become more true though. It's really hard to argue against Linux when even architecturally poor decisions are papered over by sheer force of will and investment; so in a day-to-day context Linux is often the happy path even though the UX of FreeBSD is more consistent over time.
flipped 8mo ago
Never understood why satoshi was a prime windows user.
- earthscienceman 8mo ago
  I know this comment is effectively a side tangent on a side tangent. but that was always the strangest thing to me as well. I remember in 2012 when I was debating fiddling around with Bitcoin. that was one of the things that turned me off. I was sure that there was no way something as brilliant as this was supposed to be was developed by windows user. Which surely says something about all these ideological purity tests
  - dijit 8mo ago
    Windows developers (like sysadmins) are of two kinds in my experience. People who don't understand shit about how the system behaves and are comfortable with that. "I install a package, I hit the button, it works" .. and People who understand very deeply how computers work, and genuinely enjoy features of the NT Kernel, like IOCP and the performance counters they offer to userland. What's weird to me is that the competence is bimodal; you're either in the first camp or the second. With Linux (+BSD/Solaris etc;) it's a lot more of a spectrum. I've never understood exactly why this is, but it's consistent. There's no "middle-good" Windows developer.

matheus-rr 8mo ago

The jails vs containers framing is interesting but I think it misses why Docker actually won. It wasn't the isolation tech. It was the ecosystem: Dockerfiles as executable documentation, a public registry, and compose for local dev. You could pull an image and have something running in 30 seconds without understanding anything about cgroups or namespaces. FreeBSD jails were technically solid years before Docker existed, but the onboarding story was rough. You needed to understand the FreeBSD base system first. Docker let you skip all of that. That said, I've been seeing more people question the container stack complexity recently. Especially for smaller deployments where a jail or even a plain VM with good config management would be simpler and more debuggable. The pendulum might be swinging back a bit for certain use cases.

sthuck 8mo ago
I don't think article misses it, it's exactly the point it makes
torstenvl 8mo ago
> Jails solve the isolation problem beautifully, but they don't have a native answer to shipping. That gap is real, and it's one of the main reasons the ecosystem around jails feels underdeveloped compared to Docker's world. The link literally uses the term ecosystem. Several times actually.
steve1977 8mo ago
> You could pull an image and have something running in 30 seconds without understanding anything Fixed that for you ;)
steve1977 8mo ago
Maybe FreeBSD doesn't want a jails "ecosystem"?
wolvoleo 8mo ago
Jails were never going to 'win' because they're only on an OS with 0.1% marketshare. But it's not a competition. FreeBSD does its thing and Linux does another. That's why I use FreeBSD.

NooneAtAll3 8mo ago

"failed to verify your browser"

m132 8mo ago
Getting the same thing, "Failed to verify your browser. Code 11". Some noise about WebGL in the browser console, getExtension() invoked on a null reference. LibreWolf on Linux + resist fingerprinting. Maybe opting for a better-written WAF could boost the reach?

flipped 8mo ago

Is there any technical writeup which explains how the isolation exactly works, on containers and VMs? I have always heard the high level arguments of weak isolation, same kernel, etc but never the implementation details.

flipped 8mo ago

Anyone looking to use jails might find BastileBSD helpful. It's a nice and modern jail manager.

paul_h 8mo ago
I was looking at TrueNAS CORE to see if it was a viable way to bsd-jail Linux containers. I'm really only doing this to get some protection from supply chain attacks given I'm fairly promiscuous at git-clone-and-run-a-build. Before that I was aiming for the same with Bastille and had got to the give up stage because it felt too fiddly to set up. This was a year ago. Maybe its better now
- Brian_K_White 8mo ago
  Unfortunately trunas core is dead now. zVault is a fork that is effortless to migrate in-place, but pointless because it has had no updates since the fork, it's no different from just continuing to run the derelict final version of truenas core. That just leaves xigmanas which I have not tried yet, but looks like a simpler more pure nas without the jails or vm manager, which people have told me can be filled by bastille. Or really, I'm thinking rather than even xigmanas it probably makes more sense to just use plain freebsd and never get stuck like this again. The host is stuck at 13.3. 13.3 went fully EOL December 2024. The pkg repos don't even supply packages for that any more. I have a bunch of services that run in jails, and currently I can just barely squeak by by "illegally" updating the jails to 13.5. It's not officially supported by upstream freebsd but I seem to be getting away with it for now. But even 13.5 is not going to last much longer. Then what? So really the FreeNAS ui was nice an all, but not so nice as to be worth being stuck like this now. I probably should have just skipped it and just used plain freebsd which would never have had any such problem. So maybe assuming zvault continues to not update when I finally need to move some jail past 13.5, maybe the next move is not even to xigmanas but just plain freebsd.

user3939382 8mo ago

I switched my startup’s whole infra to FreeBSD a couple months ago. Found a use after free bug that Linux’s memory management was just fine with in Gnome XSLT lib that FreeBSD properly refused. Other than that smooth sailing, jails work great. After IBM destroyed CentOS, all the Xorg politics nonsense, the list goes on with Linux, not interested. I just want something quiet and boring and stable and correctly designed. NetBSD would be my first choice but they don’t get the $ they need for drivers.

manuelabeledo 8mo ago
You don’t need to follow the news cycle to use an operating system.
- user3939382 8mo ago
  I do follow the news cycle and if I’m hearing about a software package in it, something is wrong with the people making the software and I don’t trust them. Software is an engineering discussion or at least it’s supposed to be. Here’s my community guidelines: everyone be nice and respectful engage in good faith and focus on the math. Being social is fine so long that it doesn’t become a diversion from the engineering discussion. We’re talking about code not a philosophical treatise. There are civil ways to settle disagreements. I’m so sick to death of the politics.
  - manuelabeledo 8mo ago
    It sounds like software projects are built by humans. Nothing wrong with that. Unless we’re assuming here that the BSD community is free from that.
    - user3939382 8mo ago
      The engineers I’ve worked with my whole career had no problem getting the work done without getting into giant political debates. If your politics comes before the engineering I don’t want anything to do with it.

jmclnx 8mo ago

>but they don't have a native answer to shipping I am not quite sure what this means. I had a jail a few years ago and I remember there was a utility to "back" the jail up so you could put it on another system. Are there constraints with that utility. It seemed to work, maybe I am forgetting something ? In any case I still think Jails are much better than the things Linux has. To me, it is creating a jail that is more difficult. There were ports that made it easier, I used one of them, but that port was abandoned at some point. I think it was "ezjail".

efortis 8mo ago

A two-server networking setup with VNET Jails: https://ericfortis.com/blog/freebsd-jails-network-setup https://ericfortis.com/blog/freebsd-jails-network-setup

shevy-java 8mo ago

> FreeBSD is worth a brief aside here, because it differs from Linux in a fundamental way. Linux is a kernel. What most people call "Linux" is actually that kernel combined with a GNU userland, a package ecosystem, and a set of choices that vary from distro to distro — Ubuntu, Fedora, and Arch are all running the same kernel but are meaningfully different systems underneath. It is not incorrect but ... do people really care about that distinction? Because in most situations I know of, when people refer to Linux, they almost never refer to the linux kernel. They refer to the whole operating system stack, which is typically put down via a distribution. So, Fedora, Gentoo, Arch, and so forth, are all "kind of" Linux. Barely anyone refers to the linux kernel if you look at all the discussions on the world wide web. > FreeBSD ships as a complete, coherent OS The BSDs often promote that aka "Linux is chaos, we are coherent and consistent operating system following intelligent design". Well ... this is the rise of worse is better, repeated: https://dreamsongs.com/WorseIsBetter.html https://dreamsongs.com/WorseIsBetter.html It is a great analogy that works on so many levels. Broken down to Linux versus the BSDs, I think 500 out of 500 top supercomputers running Linux kind of show which philosophy is better. The one that works better. That does not mean the BSDs are useless, but I am getting tired of the promo used by the BSD as "we are order, Linux is chaos". I compare this more to Lego building blocks. With Linux there is a stronger focus on having building blocks available. You can build up things. You have projects such as LFS/BLFS (Linux from scratch). The BSDs do not have something comparable. Which operating system is the better tinker OS? Which community created git? (Ok ok that was Linus so not really a community per se, but it originated from Linux and perhaps that was not an accident either.) > FreeBSD pioneered the practical implementation of what we now call containers. Ok great. Many modern programming languages learned from older languages; many of these older languages are dead now. You need to keep on innovating. Why is BSD so dead set on the past? > FreeBSD reached that third stage in 2000. Linux wouldn't get there until 2008 with LXC. Dumdedum ... it kind of sounds as if the FreeBSD guys are sad that Linux went on to dominate. It reminds me of NetBSD aka "we work on every toaster in the world". Then suddenly on a mailing list many years ago "wait a moment ... Linux now works on more toasters than we do". The BSDs don't seem to understand how momentum can be dominating. > Technical superiority doesn't win ecosystem wars. Linux won through a combination of fast decisions, the viral GPL licence, and strong enterprise backing from Red Hat and IBM. Then Google, Facebook, and Amazon happened — hungry for datacenters, developing tools to manage growing infrastructure at scale. They set the direction for the entire industry. Ok that flat out is incorrect. First - GPL worked well for the linux kernel, that is true. But the ecosystem includes many BSD-licences programs too, on Linux. So that explanation fails already here. LLVM has Apache License 2.0 which I kind of feel is a mix between GPL and BSD (not quite true but this is how I remember it). Then the claim is Linux won because of Red Hat. I actually find Red Hat annoying and I am glad to not depend on it. Linux is way bigger than Red Hat. IBM? I don't see what IBM did for Linux really. So that explanation also does not work. Google, Facebook, and Amazon - well, they profited from Linux. They didn't really ENABLE Linux. They would not have used Linux if Linux would have been useless. So that part came afterwards. So none of those explanations really work well here. > Linux rapidly went from "the free OS for people who can't afford commercial licences" to "the only acceptable OS for servers". That is true but not for the claims made, e. g. "because of Google". The more important question is: why did the BSDs fail? > To solve the distribution and isolation problem, Linux engineers built a set of kernel primitives (namespaces, cgroups, seccomp) and then, in a very Linux fashion, built an entire ecosystem of abstractions on top to “simplify” things No, that is also incorrect. cgroups are also very different to seccomp and the latter is even maintained independently: https://github.com/seccomp/libseccomp/releases https://github.com/seccomp/libseccomp/releases > Somehow we ended up with an overengineered mess of leaky abstractions for cloud-based, vendor-locked infrastructure. Wait a moment - he cites Docker. That's owned by a private company. What does this have to do with Linux? If company xyz does something based on FreeBSD, we would then say company xyz is responsible for FreeBSD failing or not failing? How does that work? > And this complexity has quietly reshaped how the industry thinks about deploying software. Today, if you want to run an application in a larger system, the implicit assumption is that you containerise it with Docker and orchestrate it with Kubernetes. Personally I find all this abstraction crap. With all their failures, though, things such as docker kind of present a "download this one file, then it will work fine". And that is kind of true. I saw that in in-campus use for life science faculty clusters and what not. It simplifies things for the admin there. People give a similar rationale for systemd. Personally I don't think systemd should exist, but there are people who benefit from it - that simply is a factual statement. All in all this is a very strange point of view from FreeBSD folks. At the least the NetBSD folks back then on the mailing list acknowledged the situation and then tried to find alternative strategies and in some ways succeeded (although I am not sure whether NetBSD right now runs on more toasters than Linux does - anyone has updated statistics for that?).

razighter777 8mo ago

I frequently see freeBSD jails as a highlighted feature, lauding their simplicity and ease of use. While I do admire them, there are benefits to the container approach used commonly on linux. (and maybe soon freebsd will better support OCI). First it's important to clarify "containers" are not an abstraction in the linux kernel. Containers are really an illusion achieved by use of a combination of user/pid/networking namespaces, bind mounts, and process isolation primitives through a userspace application(s) (podman/docker + a container runtime). OCI container tooling is much easier to use, and follows the "cattle not pets" philosophy, and when you're deploying on multiple systems, and want easy updates, reproducibility, and mature tooling, you use OCI containers, not LXC or freebsd jails. FreeBSD jails can't hold a candle to the ease of use and developer experience OCI tooling offers. > To solve the distribution and isolation problem, Linux engineers built a set of kernel primitives (namespaces, cgroups, seccomp) and then, in a very Linux fashion, built an entire ecosystem of abstractions on top to “simplify” things. This was an intentional design decision, and not a bad one! cgroups, namespaces, and seccomp are used extensively outside of the container abstraction. (See flatpak, systemd resource slices, firejail). By not tieing process isolation to the container abstraction, we can let non-container applications benefit from them. We also get a wide breadth of container runtime choices.

Melatonic 8mo ago
Jails have been around a long time in comparison I still see FreeBSD as being great for things like networking devices and storage controllers. You can apply a lot of the "cattle vs pets" design one level above that using VMs and orchestration tools.
znpy 8mo ago
> lauding their simplicity and ease of use Spawning a linux container is much simpler and faster than spawning a freebsd jail. I don’t know why i keep hearing about jails being better, they clearly aren’t.
- Gud 8mo ago
  Sorry what? It's a 5 line configuration file to create a FreeBSD jail.

mono442 8mo ago

> FreeBSD reached that third stage in 2000. Linux wouldn't get there until 2008 with LXC. OpenVZ and Linux vserver are older than LXC and were commonly used, though they required a patched kernel.

roryirvine 8mo ago
Yeah, both VServer and Virtuozzo were roughly contemporaneous with the initial Xen release (2001-ish?), and only a few months behind jails. But Virtuozzo was hampered by being non-free - OpenVZ wasn't open sourced until 2-3 years later, by which time the damage had been done (but, of course, Xen headed in the opposite direction at roughly the same time!) And Linux-VServer was held back by being focussed so directly at virtual hosting providers - it positioned itself against fcgi-suexec, fcgid, and php-fpm (and was much more unwieldy than any of them) rather than jails or VZ. Both were more or less ignored until the late 2000s, by which time LXC had taken a lot of mindshare - allowing the "FreeBSD was years ahead with jails" meme to take root.

deleted 8mo ago

[deleted]

lizknope 8mo ago

> Technical superiority doesn't win ecosystem wars. Linux won through a combination of fast decisions, the viral GPL licence, and strong enterprise backing from Red Hat and IBM. Then Google, Facebook, and Amazon happened — hungry for datacenters, developing tools to manage growing infrastructure at scale. They set the direction for the entire industry. In the mid 1990's the hardware driver support on Linux was much broader. Copy / paste of my comment from last year about FreeBSD I installed Linux in fall 1994. I looked at Free/NetBSD but when I went on some of the Usenet BSD forums they basically insulted me saying that my brand new $3,500 PC wasn't good enough. The main thing was this IDE interface that had a bug. Linux got a workaround within days or weeks. https://en.wikipedia.org/wiki/CMD640 https://en.wikipedia.org/wiki/CMD640 The BSD people told me that I should buy a SCSI card, SCSI hard drive, SCSI CD-ROM. I was a sophomore in college and I saved every penny to spend $2K on that PC and my parents paid the rest. I didn't have any money for that. The sound card was another issue. I remember software based "WinModems" but Linux had drivers for some of these. Same for software based "Win Printers" When I finally did graduate and had money for SCSI stuff I tried FreeBSD around 1998 and it just seemed like another Unix. I used Solaris, HP-UX, AIX, Ultrix, IRIX. FreeBSD was perfectly fine but it didn't do anything I needed that Linux didn't already do.

JCattheATM 8mo ago
> FreeBSD was perfectly fine but it didn't do anything I needed that Linux didn't already do. That's pretty much it. A lot of the people I see using a BSD these days do so because they always have and they prefer what they know, which is fine, or they just want to be contrarian. Realistically, aside from edge cases in hardware support, you can do anything you want on any modern *nix. There's not even as much of a difference between distros as people claim. All the "I want an OS that gets out of my way" and similar reasons apply to most modern well-maintained distros these days. It's more personality and familiarity than anything objective.
- lizknope 8mo ago
  I went from Slackware in 1994 to Red Hat in 1998 to Fedora when they split into Fedora and RH Enterprise. Every 2 or 3 years I will install a different distro in a VM and see "Okay, now I see what it's about." But I have no interest in switching as long as Fedora does everything I need. I don't really understand the people that distro hop. I just assume they are really young and I have work to do and a family to take care of.

smitty1e 8mo ago

Is this fair? Linux is to *BSD as VHS was to Betamax.

sidkshatriya 8mo ago
At one time this was an interesting comparison... but now Linux has gotten so much development that even if FreeBSD was Betamax and Linux VHS (in the past)... I would say that Linux is now DVD ... and FreeBSD still remains betamax. Don't get me wrong, FreeBSD is simple, elegant, consistent and well manicured. It seems to have picked up some pace again. I'm rooting for it.
- smitty1e 8mo ago
  Indeed, there is no shame in being a dirt-simple system that Just Works. Likely is has a good place at the low end. In enterprise mode, you want something like an AWS to hide the pain of those large-scale details that Linux is bringing.

epistasis 8mo ago

Long emotional rant ahead, you have been warned. The poorly thought out adoption of parts of systemd, and in particular systemd-oomd, are making me yearn for FreeBSD. For all my computing career, I'd use Unix-alike because they let me develop software by having an idea, write some code, let it run in the terminal, chain things together, and see where it failed, and iterate. Terminals let this happen much faster than clicky GUI software, and contain a log of what was happening in a terminal pane (or gnus screen or tmux pane, because usually this is happening on big servers and compute clusters rather than my terminal/laptop) I could launch a bunch of panes, have several lines of investigation going, and come back to it a day or week later when I had time because the terminal kept a log of everything that happened. And a couple years ago I started noticing that things I thought I had launched would start disappearing. At first I thought I had started accidentally mispressing a key and killing a tmux pane rather than disconnecting. But no! When I finally went back to a in-person Linux workstation and saw it happen to the entire terminal window, I knew something major had changed. It turns out that something called systemd-oomd was added to a bunch of distributions that now kills only entire cgroups of processes at a time, rather than a single process offender. So now if you want to run processes and isolate the kill zone of a process, you have to wrap every freaking subprocess in an entire systemd-run wrapper or docker wrapper. And systemd-run won't work from many contexts, such as inside a Jupyter kernel. Major breaking changes on fundamental system behavior are a huge problem these days. It's one thing to let the OS kill processes more when there's a memory issue, fine, great, go ahead. But why kill all the lightweight processes that could give feedback to a user?! And why force non-portable process launching semantics, that aren't even consistent across the entire system?!? So infuriating.

DaveCharlieLen 8mo ago

> the viral GPL licence this was the key part for Linux success, but also it was a network effect thing, if you want users for a unix they likely would use Linux, so give them a driver on Linux and redirect people. Then having the kernel binding be malleable led to more people just opensourcing the drivers if they had an actual product or just vendor the linux kernel otherwise.