8 ms·

FreeBSD 14.0 Delivering Great Performance Uplift

142 points by vermaden 3y ago

nesarkvechnep 3y ago
As expected!
PedroBatista 3y ago
FreeBSD is growing more and more on me, mostly because of Jails ( also ZFS ). Call me a boomer, but I still think "containers"/Docker is not a solid, simple and robust solution for isolation.
- pjmlp 3y ago
  We had lpar, vaults, zones, jails, but it was docker that won the mindshare, to the point many think it was the first.
  - yjftsjthsd-h 3y ago
    In fairness, I'm not aware of any preceding container/jail/whatever system that handled the packaging and distribution of software like docker did, and I suspect to most users that's more important than the containment features. Or am I forgetting something else that did that, too?
    - bayindirh 3y ago
      You actually don't need that all packaging stuff for ease-of-use much. Compile your application with required libraries to a subfolder, define the jail's root as that subfolder, and fire the app. It's similar under Linux's CGroups. You fire the binary inside another CGroup and, that application thinks that it's the only thing on the machine, and may feel a little lonely, with its own IP and isolated network stack and such.
      - yjftsjthsd-h 3y ago
        > You actually don't need that all packaging stuff for ease-of-use much. Compile your application with required libraries to a subfolder, define the jail's root as that subfolder, and fire the app. I can download and run anything in any accessible OCI registry with a single `docker run` command. The only thing I've seen that makes jails even remotely close to being as easy and quick to use... is using runj to recreate the same experience. > It's similar under Linux's CGroups. You fire the binary inside another CGroup and, that application thinks that it's the only thing on the machine, and may feel a little lonely, with its own IP and isolated network stack and such. Yeah, I know how the isolation features work, my point is that they're not nearly as important. For the vast majority of docker use, you could replace the actual container with chroot and not really make any difference, and the chroot is only to save the trouble of having to modify the application to handle host paths, not as a security barrier. Docker's killer feature isn't containers, it's images that are easy to move around and run, and to slightly lesser degree Dockerfiles that provided a standardized format to describe how to build those images. I love FreeBSD, but I think this kind of illustrates how the community has really dropped the ball on containers: Nobody cares that jails are technically superior and far more elegant under the hood, or that they've been here longer than on linux, or whether their security position is better. Docker was and is overwhelmingly better UX; all the under the hood stuff is effectively irrelevant to users.
        bayindirh 3y ago
        Well, not all shops just blindly install docker, pull a docker compose stack and call it day. Many of us out there compile their apps, stash into a cgroup or a jail and fire it. Just because mainstream developers using "that one thing", it doesn't mean that the whole world has abandoned other ways to do things. I personally doesn't pull any big software package from an OCI registry, because I don't trust the installation and don't like the reduced configurability of the packages involved. If I'm lazy for that day, I pull Debian minimal, build the thing I need, push to our local registry and call it a day. Otherwise I install that thing properly, run it under cgroups, or under its own VM if the workload warrants it, and get a mug of coffee. > Nobody cares that jails are technically superior and far more elegant under the hood... Believe me, some shops care.
    - dmpk2k 3y ago
      Zones and ZFS streams came close (but the main problem there is that ZFS assumes that ZFS streams are trusted...).
    - clan 3y ago
      Personally I find the FreeBSD package approach more elegant. We have the "pkg" package manager as standard on the system. If you want to install into a jail you simply add the -j flag. There has not been a lot of documentation on how to do service jails. Maybe because a full jail (with a full and reasonably sized OS) is so trivial to setup. But there is no denying that Docker have had great marketing and a good use case. I do however think the real reason no matter how much of a FreeBSD Jail or Solaris Zones fan boy you are - you know that Linux is the elephant in the room. That brand name alone without arguing distributions. Linux already had a huge market and mindshare compared to FreeBSD or Solaris. And when Linux got CGroups you had (within reason) feature parity. For anyone who are not daily distro switchers changing your base OS is no minor feat. So even if CGroups was not perfect the incentive to switch is not huge. But it does make me sad that Docker has become so prevalent that a Docker image is the only way some projects make releases. All too commonly without documenting the build steps. Do not be so hard on your posix friends, please!
      - nolist_policy 3y ago
        > All too commonly without documenting the build steps. That's the great thing with Docker, all you need is documented in the Dockerfile.
        clan 3y ago
        You seem to be in the "read the code" camp. I am not. Rather than using plain language and reason about dependencies, limitations, workarounds and more or less informed choices you need to infer this from the Docker file. Pray that they have left even a single little comment regarding non obvious issues. Akin to saying everything is documented in the Makefile. Why not a quick glance a main() to see if we parse any args. You can give people step by step instructions on where to go. But you empower them with a map. Documentation is hard. Quality documentation even harder. A dockerfile is a very poor substitution for me. I do not mind projects who prefer Docker and they get all my love and appreciation if they document the manual building steps. And I am totally fine by them telling me I should probably not do that. But I am seeing more projects skipping this and instead spend effort and time on using and debugging with Docker. But I am old and my beard is getting grey. I have learned to fear the programmer who tells me to read the code to understand the system. I have been told that it is concise and is good for velocity. Not everything old is good and we should move on. I for one miss the days when documentation was considered a priority.
    - pjmlp 3y ago
      Because the previous systems were about applications security, not about shipping the developer's machine into production, which is what most Docker containers are about.
  - bayindirh 3y ago
    From my understanding, Linux CGroups proved itself to be a viable alternative to jails, because, it provided "Linux inside Linux" in practice. Also, if my memory serves right, jails had some performance impact for the workload inside the jail. Docker, as you know better than me, is just a wrapper over CGroups, nothing else, plus a container image format, basically. In the end, CGroups provided what jails provided with the same and better performance, and someone leveraged that subsystem with a tool, before everyone else.
    - clan 3y ago
      Memory serves you wrong. No performance impact running inside the jail as it is basically just a jail id managed by the kernel. I have no qualms with neither Linux nor CGroups. Great technology in itself. No reason to spread FUD.
      - bayindirh 3y ago
        > Memory serves you wrong. Well, OK. > No performance impact running inside the jail as it is basically just a jail id managed by the kernel. Thanks for letting me know. > No reason to spread FUD. I was just relaying what our system and BSD guy said back in the day. I have nothing against BSD. That quote was before hardware virtualization days, BTW.
        clan 3y ago
        Virtual Machines (VMs) is 1960's technology. Started out on the mainframes. x86 virtualization came along in the 1990's. VMware Desktop everybody knows today was released around 1999. Other players have been around. This is the same year jails was introduced with FreeBSD 4. The funny thing is that jails are often referenced as "virtualization without the overhead". Jails are really fast as it is the same shared kernel. Hardware VMs have additional security advantage as a guest kernel vulnerability will (normally) not affect ofter guests. But it comes at a cost.
        bayindirh 3y ago
- tambourine_man 3y ago
  I dream of a “chroot is all you need” successor to Docker. I’m not a boomer, but I’m considered old in this industry.
  - pram 3y ago
    You can accomplish a whole lot these days with the systemd service parameters like Private* and Protect*
    - tambourine_man 3y ago
      The idea with chroot is for compatibility with anything POSIX
      - nolist_policy 3y ago
        I needed to look at your comment for 10 minutes before I got what you mean. So you want containers that run on any POSIX operating systems? You can do this with ptrace() which is in POSIX, but its going to be slow for i/o heavy applications. Unfortunately POSIX doesn't provide anything else (seccomp would already be a step up). See also proot[1], gVisor[2] and User Mode Linux[3] which all use ptrace(). [1] https://github.com/termux/proot https://github.com/termux/proot [2] https://gvisor.dev/docs/architecture_guide/platforms/#implementations https://gvisor.dev/docs/architecture_guide/platforms/#implem... [3] https://www.kernel.org/doc/html/v5.9/virt/uml/user_mode_linux.html https://www.kernel.org/doc/html/v5.9/virt/uml/user_mode_linu...
        tambourine_man 3y ago
        Sorry, here's a slightly more serious attempt at a decent comment: my hunch is that most of what people look for in Docker could be accomplished by simple file system isolation and a few env variables. Something that could be hacked together with chroot and shell scripts. This much simpler solution could be replicated in basically any decent OS and wouldn't require shipping an entire Linux kernel and deal with slow IO and the network hoops to jump through on anything that isn't Linux. The same directory of ready-made solutions that Docker enjoys today could be rebuilt uppon this simpler stack. When you realize that linking to the right libraries gets you most of the way there already, you begin to think there's a gem of a solution hidden in plain sight among all this mess.
- jlouis 3y ago
  Once you've seen the (older) alternatives, you cringe at Docker, indeed. Jails is a better solution. So is (Solaris/Illumos) zones. Nix is a great way to jump over docker entirely. You can then output a container from Nix if you need it, but you aren't limited by them at all.
  - nortonham 3y ago
    >Jails is a better solution. So is (Solaris/Illumos) zones. Can you explain why? It's not something I know a lot about so I'd be interested to learn about
- jcranmer 3y ago
  As I see it, there are roughly four main related features that Docker brings, though it is far from the only solution to bring these features: 1. The ability to create services in a clean slate environment. Unix processes tend to inherit by default most environment attributes--working directory, open file descriptors, etc.--and having a way to create a process (or group of processes) without inheriting all of that stuff is easy. 2. Virtualization/isolation in a lighter weight solution than a full virtual machine. 3. Checkpointing that allows you to consistently restart an image with a given configuration, even if you go on to horribly destroy it in tests or whatnot. 4. A language for designing how to construct these images (the Dockerfile). I suspect the popularity of Docker mostly comes from the latter, and I'd definitely like to see something like that but utilizing jails and ZFS features instead.
  - hotnfresh 3y ago
    My primary use of Docker is as an isolated-installation package manager that’s portable cross-distro and cross-distro-version with exceptionally well-documented and easy-to-test config locations and data storage paths (so much easier to be sure you’ve backed up everything you need to). Docker minus the giant well-populated well-maintained image library would be almost useless to me. [edit] one daemon improved immeasurably by Docker is Samba, of all things. The invocations are a little arcane, but once you’ve got them figured out it’s one extra option line to add a user, one line to add a share, repeat as needed. So very much better than relying on distro-magic to make it work, or, god forbid, trying to configure it manually with some config file that ends up inexplicably having no portability, or is silently ignored despite the claims of the docs, or doesn’t work at all because there’s some option commented out by default that definitely shouldn’t be. Docker forced them to finally make the “I just want to share a damn directory, to these users, either read only or read-write” use case, which is probably the vast majority of use of Samba, straightforward, concise, and reliable to configure.
- forinti 3y ago
  I'm surprised someone hasn't written a jails-like wrapper for starting containers in Linux.
  - aforwardslash 3y ago
    They have. Its called LXC. There are some differences when compared to "modern" jails (which, due to virtual network and cpu sets, are more close to solaris Zones in terms of functionality), but basically the same core functionality.
    - forinti 3y ago
      In fact it is. I've been overlooking LXC for so long. Thanks for the tip.
11Spades 3y ago
I use FreeBSD on my home server and love it. I'd daily-drive it too, if it had better WiFi support out of the box.
- Gud 3y ago
  You should try wifibox! It works surprisingly well and is very fast. Basically, it's a tiny linux running inside bhyve doing all the wifi stuff. But 14.0 has many updates to the WiFi so I'm not sure it is even needed anymore.
  - sunshine-o 3y ago
    This is brilliant ! I would also love a similar setup with OpenWRT as it comes with a familiar configuration & UI.
whalesalad 3y ago
Would have been cool to see a Linux distribution in the comparison too, just for reference.
- deleted 3y ago
  [deleted]
jabl 3y ago
Would have been more interesting to see some analysis what caused the change in performance.
- laxd 3y ago
  Lots of great work by many people. But I bet this guy and his optimizations to the vfs and locking has made a significant impact. https://github.com/freebsd/freebsd-src/commits?author=mjguzik https://github.com/freebsd/freebsd-src/commits?author=mjguzi...
bmacho 3y ago
Can I just install it on my laptop, and expect that I'll have graphics, hardware acceleration, wifi, suspend/hibernate out of the box? (There was the live hurd distribution ArchHURD that offered exactly this, I could just put the CD in, and then I had a familiar system with gnome, firefox and such.) I know there was ghostBSD and desktopBSD for this in the past, but I am not sure what is recommended today?
- Gualdrapo 3y ago
  Maybe you can try NomadBSD[0]: > NomadBSD is a persistent live system for USB flash drives, based on FreeBSD®. Together with automatic hardware detection and setup, it is configured to be used as a desktop system that works out of the box, but can also be used for data recovery, for educational purposes, or to test FreeBSD®'s hardware compatibility. Though I am not sure when/if it will be rebased to this new release of FreeBSD. [0] https://www.nomadbsd.org/ https://www.nomadbsd.org/
  - e12e 3y ago
    >> Though I am not sure when/if it will be rebased to this new release of FreeBSD. >> [0] https://www.nomadbsd.org/ https://www.nomadbsd.org/ > Current Version > 140R-20231121 Seems to be on 14?
- exitb 3y ago
  Impossibile to tell, I've tried Free, Open and NetBSD on my older ThinkPad and surprisingly OpenBSD turned out to have the best out-of-the-box compatibility. It seemed however to be the worst performance-wise with fans going crazy whenever a browser is opened up. I ended up with Linux.
- gosub100 3y ago
  If your laptop is 8-10+ years old, surprisingly yes (not sure about the suspend/hibernate part, I doubt it). I tried to recycle a friend's 2 older laptops and most everything worked out of the box. I even installed the "Chicago" Theme (looks like win95) [1] for ease of use. 1 - https://www.youtube.com/watch?v=SpaSXca-tEs&t=214s https://www.youtube.com/watch?v=SpaSXca-tEs&t=214s
dwringer 3y ago
Does anyone know what the state of FreeBSD is these days regarding the possible support of CUDA devices? It always pained me that installing it as an OS on my old hardware meant losing access to that, and a large part of the usefulness of the GPU therein.
- vermaden 3y ago
  The 'Enterprise Working Group' is also working on better support from Nvidia - also for CUDA: - https://wiki.freebsd.org/EnterpriseWorkingGroup https://wiki.freebsd.org/EnterpriseWorkingGroup From what I have seen on Twitter - the CUDA works in Linux Binary Compatibility layer on FreeBSD currently.
- sgift 3y ago
  I just did a quick search and it seems to work using the Linuxulator as a bridge - here's a tutorial that makes use of it: https://github.com/verm/freebsd-stable-diffusion https://github.com/verm/freebsd-stable-diffusion Native CUDA support doesn't seem to be on NVidias roadmap for FreeBSD, unfortunately.
abotsis 3y ago
640 cores ought to be enough for anyone!
reacharavindh 3y ago
I have always used FreeBSD to build petabyte scale ZFS-NFS servers for HPC. This latest round involved NFSv4.2 and that FreeBSD to be enrolled into RedHAT IDM for auth. It was dog slow like I could not believe, and kept spewing a slurry of errors. Without having the luxury of time to solve all the issues, we ended up going with RedHat EL 9 purely to avoid the slowness from IDM integration. I hope someone else has the time to figure out the issues with IPA integration in FreeBSD. Have a look at what it means to get FreeIPA client running on FreeBSD. https://vermaden.wordpress.com/2022/11/17/connect-freebsd-freeipa-idm/ https://vermaden.wordpress.com/2022/11/17/connect-freebsd-fr... :-( I know I sound entitled expecting a pkg install ipa-client Without doing any of the work. But, my point of writing here is merely to highlight a problem that others who are in much better position that I am to fix it can take a look.
- Khaine 3y ago
  I'm confused. Getting IPA integration looks relatively straightforward. You need to make sure that the FreeBSD packages have certain integrations enabled. If you aren't happy with the default package settings, you can raise that with the maintainer It's easy to change the package settings anyway. You can do this if you manually build them yourself using something like portmaster for a one-off server, or run your own package and build server with poudriere, if its for a cluster of servers.
- skullone 3y ago
  Are you integrating with IPA for Kerberos NFS? Or just uid/gid mappings? Wonder why the perf would be so bad over nfs after either though
  - reacharavindh 3y ago
    Sorry for the less useful rant. Yes, we are integrating with IPA for just uid/gid mappings. From the little time we got to debug before giving up, FreeBSD was fast and working as usual before IPA client enrolment was done. We tried NFSv3 (with separate lockd) and NFSv4.2 both only performing at about 20% the throughput of what we currently get out of ZFS on Linux. Both ZFS versions being the same. ZFS-2.1 It felt like we were in uncharted waters with the IPA client and gave up instead of fixing the problems. I wish there was Pkg install ipa-client
nullindividual 3y ago
I struggle to find a use for BSD outside of my opnsense VM. *BSD makes for a great gateway device (OpenBSD w/ HAProxy & CARP) and I've used it for that, but even for a webserver, I can't justify it over Linux if I want to run .NET-anything (only unofficial ports exist). What are the most popular markets for the BSDs today? Anything outside of research/edu?
- assimpleaspossi 3y ago
  Netflix video delivery. WhatsApp. And countless others.
  - ladyanita22 3y ago
    WhatsApp not anymore, and Netflix uses it only partially (only for CDN I believe, the rest of the infra is Linux)
    - assimpleaspossi 3y ago
      "Only partially" is a huge "partial" considering that it's all video content delivery everywhere Netflix operates.
  - toast0 3y ago
    WhatsApp is no longer running FreeBSD. Prior to acquisition, everything was bare metal managed hosting at SoftLayer and we had all FreeBSD except one Linux host for reasons I can't remember (maybe an expirement for calling?). After acquisition, there was a migration to Facebook hosting that included moving to Facebook's flavor of containerized Linux. Not because Linux is better, but to fit better within Facebook's operations[1], and Erlang runs on many platforms, so it was a much smaller effort to get our code running on Linux than to get FB's server management software to work for FreeBSD. Server hardware was quite a bit different, so we had no apples to apples comparisons of which OS was more efficient or whatever else. During initial migration, BEAM support of kqueue was much better than epoll, but that got worked out, and I feel like Linux's memory usage reporting is worse than FreeBSD's, but it's a weakness of both. I was never comfortable in the FB server environment, so I left in late 2019, when the FreeBSD server count was reduced to a small enough number that I ran out of things to do. [1] Much of the server team had experience with acquisitions at Yahoo! and the difficulties of making an operations team focused on one OS support acquired teams on another OS. With the many other technical and policy differences between WA and FB, eliminating the OS difference was an easy choice to reduce friction. Our host count, which was large at SoftLayer, was small at Facebook, even after factoring in increased numbers because the servers were smaller and the operations less stable.
shrubble 3y ago
I installed 14 on an HP Proliant microserver, 4GB RAM with 4x SATA and AMD dual core 1.5ghz CPU. It worked very well for the backup and light fileserving duties I have planned for it.

142 points by vermaden 3y ago

As expected!

FreeBSD is growing more and more on me, mostly because of Jails ( also ZFS ). Call me a boomer, but I still think "containers"/Docker is not a solid, simple and robust solution for isolation.

pjmlp 3y ago
We had lpar, vaults, zones, jails, but it was docker that won the mindshare, to the point many think it was the first.
- yjftsjthsd-h 3y ago
  In fairness, I'm not aware of any preceding container/jail/whatever system that handled the packaging and distribution of software like docker did, and I suspect to most users that's more important than the containment features. Or am I forgetting something else that did that, too?
  - bayindirh 3y ago
    You actually don't need that all packaging stuff for ease-of-use much. Compile your application with required libraries to a subfolder, define the jail's root as that subfolder, and fire the app. It's similar under Linux's CGroups. You fire the binary inside another CGroup and, that application thinks that it's the only thing on the machine, and may feel a little lonely, with its own IP and isolated network stack and such.
    - yjftsjthsd-h 3y ago
      > You actually don't need that all packaging stuff for ease-of-use much. Compile your application with required libraries to a subfolder, define the jail's root as that subfolder, and fire the app. I can download and run anything in any accessible OCI registry with a single `docker run` command. The only thing I've seen that makes jails even remotely close to being as easy and quick to use... is using runj to recreate the same experience. > It's similar under Linux's CGroups. You fire the binary inside another CGroup and, that application thinks that it's the only thing on the machine, and may feel a little lonely, with its own IP and isolated network stack and such. Yeah, I know how the isolation features work, my point is that they're not nearly as important. For the vast majority of docker use, you could replace the actual container with chroot and not really make any difference, and the chroot is only to save the trouble of having to modify the application to handle host paths, not as a security barrier. Docker's killer feature isn't containers, it's images that are easy to move around and run, and to slightly lesser degree Dockerfiles that provided a standardized format to describe how to build those images. I love FreeBSD, but I think this kind of illustrates how the community has really dropped the ball on containers: Nobody cares that jails are technically superior and far more elegant under the hood, or that they've been here longer than on linux, or whether their security position is better. Docker was and is overwhelmingly better UX; all the under the hood stuff is effectively irrelevant to users.
      - bayindirh 3y ago
        Well, not all shops just blindly install docker, pull a docker compose stack and call it day. Many of us out there compile their apps, stash into a cgroup or a jail and fire it. Just because mainstream developers using "that one thing", it doesn't mean that the whole world has abandoned other ways to do things. I personally doesn't pull any big software package from an OCI registry, because I don't trust the installation and don't like the reduced configurability of the packages involved. If I'm lazy for that day, I pull Debian minimal, build the thing I need, push to our local registry and call it a day. Otherwise I install that thing properly, run it under cgroups, or under its own VM if the workload warrants it, and get a mug of coffee. > Nobody cares that jails are technically superior and far more elegant under the hood... Believe me, some shops care.
  - dmpk2k 3y ago
    Zones and ZFS streams came close (but the main problem there is that ZFS assumes that ZFS streams are trusted...).
  - clan 3y ago
    Personally I find the FreeBSD package approach more elegant. We have the "pkg" package manager as standard on the system. If you want to install into a jail you simply add the -j flag. There has not been a lot of documentation on how to do service jails. Maybe because a full jail (with a full and reasonably sized OS) is so trivial to setup. But there is no denying that Docker have had great marketing and a good use case. I do however think the real reason no matter how much of a FreeBSD Jail or Solaris Zones fan boy you are - you know that Linux is the elephant in the room. That brand name alone without arguing distributions. Linux already had a huge market and mindshare compared to FreeBSD or Solaris. And when Linux got CGroups you had (within reason) feature parity. For anyone who are not daily distro switchers changing your base OS is no minor feat. So even if CGroups was not perfect the incentive to switch is not huge. But it does make me sad that Docker has become so prevalent that a Docker image is the only way some projects make releases. All too commonly without documenting the build steps. Do not be so hard on your posix friends, please!
    - nolist_policy 3y ago
      > All too commonly without documenting the build steps. That's the great thing with Docker, all you need is documented in the Dockerfile.
      - clan 3y ago
        You seem to be in the "read the code" camp. I am not. Rather than using plain language and reason about dependencies, limitations, workarounds and more or less informed choices you need to infer this from the Docker file. Pray that they have left even a single little comment regarding non obvious issues. Akin to saying everything is documented in the Makefile. Why not a quick glance a main() to see if we parse any args. You can give people step by step instructions on where to go. But you empower them with a map. Documentation is hard. Quality documentation even harder. A dockerfile is a very poor substitution for me. I do not mind projects who prefer Docker and they get all my love and appreciation if they document the manual building steps. And I am totally fine by them telling me I should probably not do that. But I am seeing more projects skipping this and instead spend effort and time on using and debugging with Docker. But I am old and my beard is getting grey. I have learned to fear the programmer who tells me to read the code to understand the system. I have been told that it is concise and is good for velocity. Not everything old is good and we should move on. I for one miss the days when documentation was considered a priority.
  - pjmlp 3y ago
    Because the previous systems were about applications security, not about shipping the developer's machine into production, which is what most Docker containers are about.
- bayindirh 3y ago
  From my understanding, Linux CGroups proved itself to be a viable alternative to jails, because, it provided "Linux inside Linux" in practice. Also, if my memory serves right, jails had some performance impact for the workload inside the jail. Docker, as you know better than me, is just a wrapper over CGroups, nothing else, plus a container image format, basically. In the end, CGroups provided what jails provided with the same and better performance, and someone leveraged that subsystem with a tool, before everyone else.
  - clan 3y ago
    Memory serves you wrong. No performance impact running inside the jail as it is basically just a jail id managed by the kernel. I have no qualms with neither Linux nor CGroups. Great technology in itself. No reason to spread FUD.
    - bayindirh 3y ago
      > Memory serves you wrong. Well, OK. > No performance impact running inside the jail as it is basically just a jail id managed by the kernel. Thanks for letting me know. > No reason to spread FUD. I was just relaying what our system and BSD guy said back in the day. I have nothing against BSD. That quote was before hardware virtualization days, BTW.
      - clan 3y ago
        Virtual Machines (VMs) is 1960's technology. Started out on the mainframes. x86 virtualization came along in the 1990's. VMware Desktop everybody knows today was released around 1999. Other players have been around. This is the same year jails was introduced with FreeBSD 4. The funny thing is that jails are often referenced as "virtualization without the overhead". Jails are really fast as it is the same shared kernel. Hardware VMs have additional security advantage as a guest kernel vulnerability will (normally) not affect ofter guests. But it comes at a cost.
        bayindirh 3y ago
tambourine_man 3y ago
I dream of a “chroot is all you need” successor to Docker. I’m not a boomer, but I’m considered old in this industry.
- pram 3y ago
  You can accomplish a whole lot these days with the systemd service parameters like Private* and Protect*
  - tambourine_man 3y ago
    The idea with chroot is for compatibility with anything POSIX
    - nolist_policy 3y ago
      I needed to look at your comment for 10 minutes before I got what you mean. So you want containers that run on any POSIX operating systems? You can do this with ptrace() which is in POSIX, but its going to be slow for i/o heavy applications. Unfortunately POSIX doesn't provide anything else (seccomp would already be a step up). See also proot[1], gVisor[2] and User Mode Linux[3] which all use ptrace(). [1] https://github.com/termux/proot https://github.com/termux/proot [2] https://gvisor.dev/docs/architecture_guide/platforms/#implementations https://gvisor.dev/docs/architecture_guide/platforms/#implem... [3] https://www.kernel.org/doc/html/v5.9/virt/uml/user_mode_linux.html https://www.kernel.org/doc/html/v5.9/virt/uml/user_mode_linu...
      - tambourine_man 3y ago
        Sorry, here's a slightly more serious attempt at a decent comment: my hunch is that most of what people look for in Docker could be accomplished by simple file system isolation and a few env variables. Something that could be hacked together with chroot and shell scripts. This much simpler solution could be replicated in basically any decent OS and wouldn't require shipping an entire Linux kernel and deal with slow IO and the network hoops to jump through on anything that isn't Linux. The same directory of ready-made solutions that Docker enjoys today could be rebuilt uppon this simpler stack. When you realize that linking to the right libraries gets you most of the way there already, you begin to think there's a gem of a solution hidden in plain sight among all this mess.
jlouis 3y ago
Once you've seen the (older) alternatives, you cringe at Docker, indeed. Jails is a better solution. So is (Solaris/Illumos) zones. Nix is a great way to jump over docker entirely. You can then output a container from Nix if you need it, but you aren't limited by them at all.
- nortonham 3y ago
  >Jails is a better solution. So is (Solaris/Illumos) zones. Can you explain why? It's not something I know a lot about so I'd be interested to learn about
jcranmer 3y ago
As I see it, there are roughly four main related features that Docker brings, though it is far from the only solution to bring these features: 1. The ability to create services in a clean slate environment. Unix processes tend to inherit by default most environment attributes--working directory, open file descriptors, etc.--and having a way to create a process (or group of processes) without inheriting all of that stuff is easy. 2. Virtualization/isolation in a lighter weight solution than a full virtual machine. 3. Checkpointing that allows you to consistently restart an image with a given configuration, even if you go on to horribly destroy it in tests or whatnot. 4. A language for designing how to construct these images (the Dockerfile). I suspect the popularity of Docker mostly comes from the latter, and I'd definitely like to see something like that but utilizing jails and ZFS features instead.
- hotnfresh 3y ago
  My primary use of Docker is as an isolated-installation package manager that’s portable cross-distro and cross-distro-version with exceptionally well-documented and easy-to-test config locations and data storage paths (so much easier to be sure you’ve backed up everything you need to). Docker minus the giant well-populated well-maintained image library would be almost useless to me. [edit] one daemon improved immeasurably by Docker is Samba, of all things. The invocations are a little arcane, but once you’ve got them figured out it’s one extra option line to add a user, one line to add a share, repeat as needed. So very much better than relying on distro-magic to make it work, or, god forbid, trying to configure it manually with some config file that ends up inexplicably having no portability, or is silently ignored despite the claims of the docs, or doesn’t work at all because there’s some option commented out by default that definitely shouldn’t be. Docker forced them to finally make the “I just want to share a damn directory, to these users, either read only or read-write” use case, which is probably the vast majority of use of Samba, straightforward, concise, and reliable to configure.
forinti 3y ago
I'm surprised someone hasn't written a jails-like wrapper for starting containers in Linux.
- aforwardslash 3y ago
  They have. Its called LXC. There are some differences when compared to "modern" jails (which, due to virtual network and cpu sets, are more close to solaris Zones in terms of functionality), but basically the same core functionality.
  - forinti 3y ago
    In fact it is. I've been overlooking LXC for so long. Thanks for the tip.

11Spades 3y ago

I use FreeBSD on my home server and love it. I'd daily-drive it too, if it had better WiFi support out of the box.

Gud 3y ago
You should try wifibox! It works surprisingly well and is very fast. Basically, it's a tiny linux running inside bhyve doing all the wifi stuff. But 14.0 has many updates to the WiFi so I'm not sure it is even needed anymore.
- sunshine-o 3y ago
  This is brilliant ! I would also love a similar setup with OpenWRT as it comes with a familiar configuration & UI.

whalesalad 3y ago

Would have been cool to see a Linux distribution in the comparison too, just for reference.

deleted 3y ago
[deleted]

jabl 3y ago

Would have been more interesting to see some analysis what caused the change in performance.

laxd 3y ago
Lots of great work by many people. But I bet this guy and his optimizations to the vfs and locking has made a significant impact. https://github.com/freebsd/freebsd-src/commits?author=mjguzik https://github.com/freebsd/freebsd-src/commits?author=mjguzi...

bmacho 3y ago

Can I just install it on my laptop, and expect that I'll have graphics, hardware acceleration, wifi, suspend/hibernate out of the box? (There was the live hurd distribution ArchHURD that offered exactly this, I could just put the CD in, and then I had a familiar system with gnome, firefox and such.) I know there was ghostBSD and desktopBSD for this in the past, but I am not sure what is recommended today?

Gualdrapo 3y ago
Maybe you can try NomadBSD[0]: > NomadBSD is a persistent live system for USB flash drives, based on FreeBSD®. Together with automatic hardware detection and setup, it is configured to be used as a desktop system that works out of the box, but can also be used for data recovery, for educational purposes, or to test FreeBSD®'s hardware compatibility. Though I am not sure when/if it will be rebased to this new release of FreeBSD. [0] https://www.nomadbsd.org/ https://www.nomadbsd.org/
- e12e 3y ago
  >> Though I am not sure when/if it will be rebased to this new release of FreeBSD. >> [0] https://www.nomadbsd.org/ https://www.nomadbsd.org/ > Current Version > 140R-20231121 Seems to be on 14?
exitb 3y ago
Impossibile to tell, I've tried Free, Open and NetBSD on my older ThinkPad and surprisingly OpenBSD turned out to have the best out-of-the-box compatibility. It seemed however to be the worst performance-wise with fans going crazy whenever a browser is opened up. I ended up with Linux.
gosub100 3y ago
If your laptop is 8-10+ years old, surprisingly yes (not sure about the suspend/hibernate part, I doubt it). I tried to recycle a friend's 2 older laptops and most everything worked out of the box. I even installed the "Chicago" Theme (looks like win95) [1] for ease of use. 1 - https://www.youtube.com/watch?v=SpaSXca-tEs&t=214s https://www.youtube.com/watch?v=SpaSXca-tEs&t=214s

dwringer 3y ago

Does anyone know what the state of FreeBSD is these days regarding the possible support of CUDA devices? It always pained me that installing it as an OS on my old hardware meant losing access to that, and a large part of the usefulness of the GPU therein.

vermaden 3y ago
The 'Enterprise Working Group' is also working on better support from Nvidia - also for CUDA: - https://wiki.freebsd.org/EnterpriseWorkingGroup https://wiki.freebsd.org/EnterpriseWorkingGroup From what I have seen on Twitter - the CUDA works in Linux Binary Compatibility layer on FreeBSD currently.
sgift 3y ago
I just did a quick search and it seems to work using the Linuxulator as a bridge - here's a tutorial that makes use of it: https://github.com/verm/freebsd-stable-diffusion https://github.com/verm/freebsd-stable-diffusion Native CUDA support doesn't seem to be on NVidias roadmap for FreeBSD, unfortunately.

abotsis 3y ago

640 cores ought to be enough for anyone!

reacharavindh 3y ago

I have always used FreeBSD to build petabyte scale ZFS-NFS servers for HPC. This latest round involved NFSv4.2 and that FreeBSD to be enrolled into RedHAT IDM for auth. It was dog slow like I could not believe, and kept spewing a slurry of errors. Without having the luxury of time to solve all the issues, we ended up going with RedHat EL 9 purely to avoid the slowness from IDM integration. I hope someone else has the time to figure out the issues with IPA integration in FreeBSD. Have a look at what it means to get FreeIPA client running on FreeBSD. https://vermaden.wordpress.com/2022/11/17/connect-freebsd-freeipa-idm/ https://vermaden.wordpress.com/2022/11/17/connect-freebsd-fr... :-( I know I sound entitled expecting a pkg install ipa-client Without doing any of the work. But, my point of writing here is merely to highlight a problem that others who are in much better position that I am to fix it can take a look.

Khaine 3y ago
I'm confused. Getting IPA integration looks relatively straightforward. You need to make sure that the FreeBSD packages have certain integrations enabled. If you aren't happy with the default package settings, you can raise that with the maintainer It's easy to change the package settings anyway. You can do this if you manually build them yourself using something like portmaster for a one-off server, or run your own package and build server with poudriere, if its for a cluster of servers.
skullone 3y ago
Are you integrating with IPA for Kerberos NFS? Or just uid/gid mappings? Wonder why the perf would be so bad over nfs after either though
- reacharavindh 3y ago
  Sorry for the less useful rant. Yes, we are integrating with IPA for just uid/gid mappings. From the little time we got to debug before giving up, FreeBSD was fast and working as usual before IPA client enrolment was done. We tried NFSv3 (with separate lockd) and NFSv4.2 both only performing at about 20% the throughput of what we currently get out of ZFS on Linux. Both ZFS versions being the same. ZFS-2.1 It felt like we were in uncharted waters with the IPA client and gave up instead of fixing the problems. I wish there was Pkg install ipa-client

nullindividual 3y ago

I struggle to find a use for BSD outside of my opnsense VM. *BSD makes for a great gateway device (OpenBSD w/ HAProxy & CARP) and I've used it for that, but even for a webserver, I can't justify it over Linux if I want to run .NET-anything (only unofficial ports exist). What are the most popular markets for the BSDs today? Anything outside of research/edu?

assimpleaspossi 3y ago
Netflix video delivery. WhatsApp. And countless others.
- ladyanita22 3y ago
  WhatsApp not anymore, and Netflix uses it only partially (only for CDN I believe, the rest of the infra is Linux)
  - assimpleaspossi 3y ago
    "Only partially" is a huge "partial" considering that it's all video content delivery everywhere Netflix operates.
- toast0 3y ago
  WhatsApp is no longer running FreeBSD. Prior to acquisition, everything was bare metal managed hosting at SoftLayer and we had all FreeBSD except one Linux host for reasons I can't remember (maybe an expirement for calling?). After acquisition, there was a migration to Facebook hosting that included moving to Facebook's flavor of containerized Linux. Not because Linux is better, but to fit better within Facebook's operations[1], and Erlang runs on many platforms, so it was a much smaller effort to get our code running on Linux than to get FB's server management software to work for FreeBSD. Server hardware was quite a bit different, so we had no apples to apples comparisons of which OS was more efficient or whatever else. During initial migration, BEAM support of kqueue was much better than epoll, but that got worked out, and I feel like Linux's memory usage reporting is worse than FreeBSD's, but it's a weakness of both. I was never comfortable in the FB server environment, so I left in late 2019, when the FreeBSD server count was reduced to a small enough number that I ran out of things to do. [1] Much of the server team had experience with acquisitions at Yahoo! and the difficulties of making an operations team focused on one OS support acquired teams on another OS. With the many other technical and policy differences between WA and FB, eliminating the OS difference was an easy choice to reduce friction. Our host count, which was large at SoftLayer, was small at Facebook, even after factoring in increased numbers because the servers were smaller and the operations less stable.

shrubble 3y ago

I installed 14 on an HP Proliant microserver, 4GB RAM with 4x SATA and AMD dual core 1.5ghz CPU. It worked very well for the backup and light fileserving duties I have planned for it.