30 ms·

We're migrating many of our servers from Linux to FreeBSD

445 points by NexRebular 5y ago

bell-cot 5y ago
tl;dr - FreeBSD has ample nice features for their use case, and is considerably simpler. Linux has loads of unneeded (for their use case) features, and so many cooks in the kitchen that the ongoing cognitive load (to keep track of the features and complexity and changes) looks worse than the one-time load of switching over to FreeBSD.
blakesterz 5y ago
"Some time ago we started a complex, continuous and not always linear operation, that is to migrate, where possible, most of the servers (ours and of our customers) from Linux to FreeBSD." I don't really disagree with any of the stated reasons, but I also didn't see a reason that would make me even consider making the move with our servers, or even bother with some small number of servers. At least for me, I'd need a bunch of REALLY GOOD reasons to consider a move like that. A huge cost savings AND some huge time savings in the future might do it.
- johnklos 5y ago
  Some people see the bigger picture and recognize that a medium amount of work now is better than lots of small amounts of work stretched over many years. Likewise, some people and many businesses see the immediate now and aren't always the best at planning for long term, and/or are overly optimistic that their pain points will eventually be fixed.
  - pm90 5y ago
    > Likewise, some people and many businesses see the immediate now and aren't always the best at planning for long term, and/or are overly optimistic that their pain points will eventually be fixed. But that's the thing. The pain points mentioned in this article aren't really that strong to need to ditch the OS and move to a new one. These kinds of decisions have huge tradeoffs. One could argue, in fact, that moving to a non-traditional OS will make it much harder to hire experts or hand off the system to another team in the future.
    - djbusby 5y ago
      What? FreeBSD as a non-traditonal OS? I must disagree, FreeBSD may not be as common as Windows or Linux but it's not like Plan9 (from outer space) or BeOS.
    - washadjeffmad 5y ago
      I think of it this way: If you interpret "rolling stones gather no moss" to mean you've always got to keep pushing forward or you'll become obsolete, choose Linux. If you venerate moss as proof of the stability granted by doing the same thing simply and perfectly, you're probably already using FreeBSD. After CentOS Stable was cancelled, I migrated a number of our platforms to FreeBSD because it met our needs and I enjoyed working on it. No surprises, nothing breaking, and most importantly, no drama.
- raverbashing 5y ago
  I agree. Nothing there seems to deliver explicit customer value when switching to FreeBSD. How will switching help you deliver your service? Or is it just a "nice to have thing"?
  - linksnapzz 5y ago
    Under what circumstances does the choice of backend OS ever deliver "explicit customer value", so much so that the customer would care about said choice?
    - lp0_on_fire 5y ago
      Normally the customer doesn't care. They will care, however, if something goes wrong during the migration or some unforeseen issue comes up later that degrades their experience.
    - brimble 5y ago
      It is definitely the case that underlying architecture, certainly all the way down to the OS, can be the difference between "I can create, test, and deploy this feature in a week, and it will be rock-solid" and "it'll take months and still fail on some edge cases—and fixing those is not remotely in our budget".
    - nine_k 5y ago
      I'd hazard to say that it delivers customer value via two avenues: (1) better SLAs, and (2) lower expenses. If your backend OS starts to run software more efficiently, cost less to host, has less maintenance downtime, has fewer security incidents, has fewer crashes, etc, having changed to it has produced customer value.
    - Twirrim 5y ago
      > Under what circumstances does the choice of backend OS ever deliver "explicit customer value", so much so that the customer would care about said choice? Is the service working, is it not? (are there show stopping OS issues?) Are the performance characteristics we need there, or not? Is the service secure or not? There's a whole lot of things right through that entire space where the choice of operating system can make quite a fundamental difference. As a general statement, I wouldn't want to make this kind of a migration without something bordering on a killer feature that wasn't possible otherwise, or some fundamental driver problems (I've replaced Linux with FreeBSD on border devices in the past, due to particular issues with Linux in ways I couldn't afford to have keep biting me)
- toast0 5y ago
  I agree that the stated reasons don't sound very compelling. Maybe in aggregate, but not individually. But they left out one of the bigger reasons, IMHO. FreeBSD doesn't tend to churn user (admin) facing interfaces. This saves you time, because you still use ifconfig to configure interfaces, and you still use netstat to look at network statistics, etc; so you don't have to learn a new tool to do the same thing but differently every couple of years. Sure, there's three firewalls, but they're the same three firewalls since forever.
  - crazy_hombre 5y ago
    ifconfig/netstat was deprecated more than a decade ago, that's more than a couple years don't you think?
    - NoSorryCannot 5y ago
      Since it was just an example, I don't think refuting this particular item will nullify the opinion. The idea, I think, is that there are always more pieces in a state of deprecation and replacement at any given time in Linux land than in FreeBSD land.
      - phkahler 5y ago
        I think that's just due to the pace of development. The BSDs are resource constrained, so they have to pick and choose what to work on. That is both a good thing and a bad thing. Here the benefit is less churn. On the downside, they're just catching the Wayland train recently. On the up side, by catching it late they didn't suffer a lot of the growing pains.
    - comex 5y ago
      Deprecated on Linux. But I for one can’t consign them to the dustbin of my memory, because on my Mac, they are not deprecated, while the `ip` command that replaces them on Linux does not exist. With this part of macOS being derived from FreeBSD, I don’t know whether that makes FreeBSD a savior or a villain. Personally I blame all of the major Unix-derived operating systems (Linux, macOS, BSDs), as none of them show any interest in standardizing any APIs or commands invented this millennium. The subset that’s common to all of them is frozen in time, and is slowly being replaced by new bits that aren’t. From epoll/kqueue to eBPF, containers/jails to seccomp/pledge, DBus/XPC to init systems… from low-level network configuration (ifconfig, ip) to high-level network configuration (whatever that is this month).
betaby 5y ago
TL;DR - for no compelling reasons.
- eatonphil 5y ago
  I don't think that's fair. Maybe half of their reasons are more on the subjective side but half of them are actual technical choices like wanting ufs/zfs and jails.
  - ianai 5y ago
    It’s even made easy to TLDR by their reasons being headlined in larger font and bold from the rest of the text. My question is how well moving their systems will shake out longer term.
  - betaby 5y ago
    zfs on FreeBSD is from the same sources as in Linux. There is not a single word in the article why to chose jails. Article is 90s style rant, which is not a bad thing.
    - zinekeller 5y ago
      > zfs on FreeBSD is from the same sources as in Linux. You should know that due to license incompatibilities (CDDL and GPLv2), it's not really as smooth as the BSD integration (I wonder how Canonical avoids this issue). You know that OpenZFS's codebase is a monorepo (for the most part) but the in-kernel implementation is vastly different.
      - mnd999 5y ago
        AFAIK, Canonical are just hoping nobody sues them. I use Zfs as my boot file system on FreeBSD and Arch Linux and both work great. Linux was a bit of a faff to set up though, FreeBSD was easy.
        quags 5y ago
AnonHP 5y ago
I have a tangential question on this part: > I sometimes experienced severe system slowdowns due to high I/O, even if the data to be processed was not read/write dependent. On FreeBSD this does not happen, and if something is blocking, it blocks THAT operation, not the rest of the system. I’ve seen this for a long time in Windows, where any prolonged I/O brings the entire system down to its knees. But it also seems to affect macOS (which is based on FreeBSD) as a system, though it’s not as bad as on Windows. Has Windows improved on this over the years? I’m unable to tell.
- philliphaydon 5y ago
  I have no idea how operating systems work, but at a wild guess. If you're running the OS off the same disk thats threashing the io, and, its queued too many read/write operations to handle the reads for the OS? And maybe FreeBSD is just so small it effectively caches itself into memory and doesn't need the IO so it appears to function still? Curious to know the reason.
  - philliphaydon 5y ago
    I guess based on the downvotes I’m wrong but they also don’t know? Reason I want to know is when I tried running ubuntu off a spinning disk and wanted to move some games to the drive it would transfer about 1gb then lock up. Didn’t happen with an SSD.
    - loeg 5y ago
      Yeah, in general wild speculation (especially if it happens to be incorrect, as it is in this case) is discouraged on this forum.
- bopbeepboop 5y ago
- yehNonsense0 5y ago
  It’s either the disk is the bottleneck or the GUI process taking a back seat to the OS prioritizing the “real work” requested. Computers get faster, we throw more at them. Physics is still the law of the land. Truh truh truh trade offfss; in computer eng; truh truh truh trade offs sorry Bowie
mbreese 5y ago
I ran a FreeBSD ZFS NFS server for a cluster for quite a while. I loved it. It was simple and stable. The thing that led me away from FreeBSD (aside from IT not being happy with an "alternative" OS), was that I needed a clustered filesystem. We outgrew the stage where I was comfortable with a single node and where upgrading storage meant a new JBOD. Are there any FreeBSD-centric answers to Ceph or Gluster or Lustre or BeeGFS?
- arminiusreturns 5y ago
  I haven't checked on it in a while but dragonfly has HAMMER2, the last docs https://gitweb.dragonflybsd.org/dragonfly.git/blob/57614c517203403e7fe4a34c370c7151ba52c539:/sys/vfs/hammer2/DESIGN https://gitweb.dragonflybsd.org/dragonfly.git/blob/57614c517... Might be a future alternative in the space.
  - loeg 5y ago
    HAMMER2 is not yet clustered, as far as I know. They're still working on single-node functionality (I think). https://gitweb.dragonflybsd.org/dragonfly.git/blob_plain/HEAD:/sys/vfs/hammer2/DESIGN https://gitweb.dragonflybsd.org/dragonfly.git/blob_plain/HEA... was last updated in 2018, and at the time much of the clustering logic is described as "under development" or "not specced."
- prakhunov 5y ago
  This changed a couple of years ago but GlusterFS does run on FreeBSD now. It's not BSD centric but it works. https://www.freshports.org/net/glusterfs/ https://www.freshports.org/net/glusterfs/
- mikehotel 5y ago
  The handbook and wiki has info on FreeBSD Highly Available Storage: https://docs.freebsd.org/en/books/handbook/disks/#disks-hast https://docs.freebsd.org/en/books/handbook/disks/#disks-hast https://wiki.freebsd.org/HAST https://wiki.freebsd.org/HAST
- diekhans 5y ago
johnklos 5y ago
This articulates most of my frustrations with the Linux world. Some of the distros are very good, but some of us who have work to do cringe at the thought of bringing up newer versions of an OS just to check all the things that've broken and changed needlessly.
- briffle 5y ago
  I hear people always complaining about 'needless' changes like systemd over init. But I sure like how my modern database servers reboot in less than 30 seconds, vs 5-12 minutes when they were running RHEL 5. (yes, some of that is because we moved from BIOS to UEFI)
  - zinekeller 5y ago
    Hmm, why do I feel that a) the older servers verifies memory banks fully before boot (which takes up minutes) and b) the change to solid-state media has the biggest impact and not on SystemD. Can you confirm that at least a) is not the reason for slow boot times?
    - antoinealb 5y ago
      I know that anecdote is not data, but I had an Arch Linux laptop when the distribution moved from Init scripts (SysV init ?) to systemd, and the boot time was easily cut in three, going from 30s to 10s, on exactly the same laptop. Of course switching from HDD to SSD was a huge improvement, but don't discount what systemd's efficient boot parallelism was able to achieve.
      - trasz 5y ago
        It’s worth keeping in mind that the old Linux rc scripts where quite mediocre compared to their BSD counterparts. They didn’t even have a dependency mechanism. So, it’s fine to compare sysv Linux scripts to systemd, but please don’t extrapolate that to other systems.
Cloudef 5y ago
Isnt bsd's tcp stack single threaded?
- rwaksmunski 5y ago
  Netflix is pushing 400Gbit/s of TLS traffic per server with 60% CPU load. WhatsApp was doing millions of concurrent TCP connections per server. FreeBSD's networking has been multi-threaded for a long time now.
  - hestefisk 5y ago
    Esp thanks to in-kernel TLS, which is fantastic.
  - technofiend 5y ago
    As seen most recently here https://news.ycombinator.com/item?id=28584738 https://news.ycombinator.com/item?id=28584738
- Beermotor 5y ago
  It has been multi-threaded since at least the early 2000s and most of the work for handling scaling beyond 32 cores was completed in 2012. https://www.cl.cam.ac.uk/teaching/1516/ConcDisSys/2015-ConcurrentSystems-1B-L8-handout.pdf https://www.cl.cam.ac.uk/teaching/1516/ConcDisSys/2015-Concu...
  - trasz 5y ago
    Well, to be honest a whole lot has been done to FreeBSD network stack scalability quite recently (14-CURRENT), eg introduction of epoch(9) and the routing nexthop patches.
- hyperionplays 5y ago
  It's multithreaded. I run FRR Routing punching 400gbit+ @ 75% CPU load on FreeBSD 12. On the same hardware, Debian fell over at 1.8gbit.
CyberRabbi 5y ago
To be fair all of these reasons come down to personal preference (sans the TCP performance claim). E.g. he prefers FreeBSD’s performance monitoring tools to Linux’s monitoring tools, or he prefers FreeBSD’s user land to Linux’s user land. That’s fine but it’s not very persuasive.
- rwaksmunski 5y ago
  vmstat -z gives me counters of kernel limit failures on FreeBSD. Very useful when debugging errors in very high performance environments. Anybody knows what the Linux equivalent is? Say I need to know how many times file descriptor/network socket/firewall connection state/accepted connection limits were reached?
  - CyberRabbi 5y ago
    If one doesn’t exist out of the box you can relatively simply roll your own using bcc https://github.com/iovisor/bcc https://github.com/iovisor/bcc.
marcodiego 5y ago
Linux took many markets. The HPC, for example, has been 100% linux in TOP500 for a few years already. Monopoly by FLOSS is still monopoly. Healthy competition is good for users and forces options to improve, see LLVM vs GCC. To sum up: healthy FLOSS competition is welcome and needed.
- pjmlp 5y ago
  Agreed, if UNIX as concept is ever to evolve, it cannot be bound at UNIX === Linux that many now seem to consider.
  - Shared404 5y ago
    I think it's somewhat to late for UNIX to evolve in general. There's too many decades of cruft and backwards compatibility built up. Afaict, most interesting new OS's being built right now are similar to UNIX, but very explicitly not.
    - trenchgun 5y ago
      GNU is not UNIX, though.
    - marcodiego 5y ago
      I don't know about UNIX per se, but consider Linux and MacOS progress in the last two decades. MacOS showed that it is possible for UNIX to be successful on the desktop. During the same period, Linux scaled from embedded computers and smartphones to supercomputers and servers. In terms of innovations, I'd bet MacOS has evolved too. Although "logical partitions"-like solutions were already known for some time, Linux made it widespread through containers; io_uring allows high throughput syscall-less zero-copy data transfer and futex2 allows to implement NT synchronization semantics that are very common in game development. All that ignoring just how much the desktop changed! The UNIX children are definitely not sitting still.
      - Shared404 5y ago
gtsop 5y ago
It feels like the title is wrong. Instead of saying "Linux is bad because I encountered X problem in production, which would have been prevented by BSD" the author goes on to list why BSD is better in general outside his specific use case. Nothing wrong with the comparison probably, but I got the impression the author just really wanted to do the migration and found some reasons to do so, without actually needing it. Nothing wrong with that as well. It's just the expectations set by the title that are off
- eatonphil 5y ago
  Neither the HN title nor the blog title is saying Linux is bad though. The title seems pretty in line with the article to me.
  - gtsop 5y ago
    The blog titles says "why we are migrating...". I didn't get any of that. I got " why I like BSD, and thus I am migrating"
- tomxor 5y ago
  I had a similar feeling, throughout reading the post I wanted to know what the specific issues were that made BSD better suited, it's all been too abstracted. I think many of us (including me) have a tendency to try to quickly generalise our experiences, even when it's not appropriate - and when we go onto explain things to others without the original context it can sound too abstract or come across as evangelical. Either way, it loses meaning without real examples.
ianai 5y ago
“ The system is consistent - kernel and userland are created and managed by the same team” Their first reason is really saying a lot but with few words. For one, there’s no systemd. The init system is maintained alongside the entire rest of the system which adds a lot of consistency. The documentation for FreeBSD is also almost always accurate and standard. Etc etc I think you also largely don’t need a docker or etc in it since jails have been native to the OS for decades. I’d want to do some cross comparison first though before committing to that statement. Shouldn’t be lost that the licensing is also much friendlier to business uses. There’s afaik no equivalent to rhel, for that matter. This goes both ways though as how would you hire a FreeBSD admin based on their resume without a rhce-like FreeBSD certification program? Edit-I’ll posit that since FreeBSD is smaller an entity wishing to add features to the OS might face either less backlash or at least enjoy more visibility from the top developers of the OS. Linus, for instance, just has a larger list of entities vying for his attention on issues and commits.
idoubtit 5y ago
The reasons are, for a large part, not on the technical side. I was surprised, because this this a lot of work for little visible gain. Here are the reasons, slightly abbreviated: > The whole system is managed by the same team Mostly philosophical. > FreeBSD development is less driven by commercial interests. Mostly philosophical. > Linux has Docker but FreeBSD has jails! IMO, this comparison is a mistake. In the Linux world, systemd's nspawn is very similar to Jails. It's a glorified chroot, with security and resource management. All the systemd tools work seemlessly with nspawn machines (e.g. `systemctl status`). Containers à la Docker are a different thing. BTW, I thought the last sentence about security issues with Docker images was strange. If you care about unmaintained images, build them yourself. On the other side, the FreeBSD official documentation about Jails has a big warning that starts with "Important: the official Jails are a powerful tool, but they are not a security panacea." > Linux has no official support for zfs and such Fair point, though I've heard about production systems with zfs on Linux. > The FreeBSD boot procedure is better than grub. YMMV > FreeBSD's network is more performant. Is there some conclusive recent benchmark about this. The post uses a 2014 post about ipv6 at Facebook, which I think is far from definitive today. Especially more since it "forgot" to mention that Facebook intended to enhance the "Linux kernel network stack to rival or exceed that of FreeBSD." Did they succeed over these 8 years ? > Straightforward system performance analysis The point is not about the quality of the tools, but the way each distribution packages them. Seems very very low impact to me. > FreeBSD's Bhyve against Linux's KVM The author reluctantly admits that KVM is more mature.
- LeonenTheDK 5y ago
  I have essentially the same take. The sysadmin at my company prefers FreeBSD for all these reasons (as such that's what we're running), and he's engaged me a tonne about FreeBSD but all I see is an operating system that's just as good as the other mainstream server Linux distributions. Except now we've got a system that's more difficult to hire for. "Any competent admin can learn it easily" is something I've been told but how many will want to when they could easily go their whole career without encountering it again? I like your point about Docker vs Jails, I haven't seen it discussed like that before. I keep hearing Jails are more secure than anything else, I'll have to read more into it. As far as the networking goes, I haven't seen any recent benchmarks to substantiate those claims either. However, considering Netflix uses FreeBSD on their edge nodes and has put a lot of work into the upstream to improve networking (among other things), it wouldn't surprise me if it's technically superior to the Linux stack. Clearly though Linux's networking isn't an issue for most organizations. And regarding ZFS, ZFS on Linux and FreeBSD's ZFS implementation are now one and the same. It would be nice to see some of the big distributions(or even the Linux kernel) integrate it more directly. This is probably a solid point in favor of FreeBSD, but it's not like it doesn't work in Linux. I'm not a systems guy, so I'm probably out of the loop on this, but Proxmox is the only distribution I've seen with ZFS as a filesystem out of the box, but I don't know how much production use Proxmox sees. I only run it on my home server. All that to basically say, I like FreeBSD conceptually. I'm just still not convinced that it's doing enough things better to warrant using it over a common Linux distribution for general computing purposes.
acatton 5y ago
Funny enough, I decided to play with FreeBSD for personal projects in 2020. I gave up and I am reverting all my servers to Linux in 2022, for the opposite of the reasons mentioned in this article. * Lack of systemd. Managing services through shell scripts is outdated to me. It feels very hacky, there is no way to specify dependencies, and auto-restarts in case of crashes. Many FreeBSD devs praise launchd, well... systemd is a clone of launchd. * FreeBSD jail are sub-optimal compared to systemd-nspawn. There are tons of tools to create freebsd jails (manually, ezjail, bastillebsd, etc…) half of them are deprecated. At the end all of your jails end up on the same loopback interface, making it hard to firewall. I couldn't find a way to have one network interface per jail. With Linux, debootstrap + machinectl and you're good to go. * Lack of security modules (such as SELinux) -- Edit: I should have written "Lack of good security module" * nftables is way easier to grasp than pf, and as fast as pf, and has atomic reloads.
- DarylZero 5y ago
  - DarylZero 5y ago
    Haters be voting.
- SpaceInvader 5y ago
  Regarding jails - I do use separate loopback device per jail, plus pf with nat. No issues with firewalling.
  - densone 5y ago
    Same .. I don’t use the separate loop back. Just firewall what I need to firewall .
- fullstop 5y ago
  I've grown to love systemd. It solves a lot of my problems with init scripts, particularly the ones involving environment / PATH at boot time. I've made init scripts for things before which work when they are invoked manually, but not at boot time because PATH was different. With systemd I am confident that if it works through systemctl it will work at boot. Maybe I am not tuning Linux appropriately, but I have been in situations where a Linux system is overwhelmed / overloaded and I am unable to ssh to it. I have never had that experience with FreeBSD -- somehow ssh is always quick and responsive even if the OS is out of memory. Most of the systems that I deal with are Linux, but I still have a few FreeBSD systems around and they are extraordinarily stable.
TurningCanadian 5y ago
"Btrfs is great in its intentions but still not as stable as it should be after all these years of development." may have been true years ago, but doesn't seem to be anymore.
- csdvrx 5y ago
  Give it another 10 years, and we may get a stable alternative to ZFS that will live in the kernel tree.
  - dralley 5y ago
    And maybe it will be bcachefs :)
- traceroute66 5y ago
  How's the old RAID56 problem in BTRFS coming on ? ;-)
  - TurningCanadian 5y ago
    Pick any given feature and a FS may or not support it well. RAID56 did have a major update, but still has the write hole and isn't recommended for production. I think the point still stands though. There has been lots of stabilization work done to BTRFS, and anyone using production-recommended features should consider the filesystem stable.
themerone 5y ago
The Wireguard debacle scared me off from FreeBSD. It seems they put too much trust in committers and don't have a solid enough review process.
- loeg 5y ago
  That Ars article is so distorted that it should best be seen as creative fiction.
  - themerone 5y ago
    Do you have a link to the real story?
  - danachow 5y ago
    Who mentioned anything about an Ars article? I read the whole debacle on mailing lists and Twitter. That was enough to form my own conclusions and I did not come away from that impressed with the FreeBSD development environs. It definitely flys in the face that somehow FreeBSD upholds technical excellence above business interests as is being claimed in the article.
    - kevans91 5y ago
      It was quite blown out of proportion even outside of the published articles.
- Melatonic 5y ago
  What was the debacle?
  - themerone 5y ago
    FreeBSD 13 came very close to shipping with a WireGaurd implementation with many bugs an vulerabilites that were quickly identified by the creator of the WireGaurd prototocal shortly after learning about the update. Attempts were made to fix it, but they eventually decided to ship 13.0 without wiregaurd. It was very policitcal because the company that sponsored the development had already promised the feature to customers.
tombert 5y ago
I don't have enough experience with FreeBSD (outside of FreeNAS seven years ago), but I've never had any success getting it to run on a laptop. Every time I've tried installing it on a laptop I get issues with either the WiFi card not working, issues with the 3D accelerator card not working at all, or the lid-close go to sleep functionality not working. I've been using Linux since I was a teenager, so it's not like I am a stranger to fixing driver issues, but it seemed like no amount of Googling was good enough for me fix these problems (googling is much harder when you don't have functioning wifi). As a result I've always just stuck with Linux (or macOS semi-recently, which I suppose is kind of BSD?).
- sandworm101 5y ago
  >> lid-close go to sleep functionality I've read hundreds of post surrounding this issue. I've used laptops for decades but I don't think I have ever used this feature. It doesn't take even a second to hit a couple keys and sleep a laptop. The only feature lower on my priority list would be syncing the RGB keyboard to soundcloud. But that's just me. Evidently the lid-close-sleep thing is of vital import to millions of laptop users. It's one of those things where I just shake my head in bewilderment.
  - trasz 5y ago
    I use it all the time; it boils down to a single line in sysctl.conf: hw.acpi.lid_switch_state=“S3”
  - Melatonic 5y ago
    One of the features I always actually make sure to turn off - I am in the same boat and there are many times when I want to close the lid of my laptop but still have some stuff chugging along. Saves a lot of battery that way!
  - tombert 5y ago
    I mean, sure, I'm an engineer, I'm sure I can figure out a workaround, but I like the feature of going to sleep on lid close. It's a feature I use, and I couldn't get it working in FreeBSD.
lazyant 5y ago
FreeBSD is a nicer, more logical Unix than Linux in general. Now as soon as you have a package or hardware that you want to use and it's not supported by FreeBSD let us know how that goes.
- edgyquant 5y ago
  It may be a more “logical unix” but most engineers (including me) don’t care about that. I started using Linux in the mid 2000s and had never heard of unix at the time and only discovered it by reading about Linux.
densone 5y ago
First off FreeBSD FTW. I use it everywhere over Linux now for the first time in 25 years and couldn’t be happier. My only wish is that BSD had a better non-CoW file system. Databases and Blockchains are already CoW so it does irk me slightly to use zfs for them. That being said, I’ve never had a problem because of it.
- chalst 5y ago
  If your complaint about UFS is the lack of journalling, you might be interested in https://docs.freebsd.org/en/articles/gjournal-desktop/ https://docs.freebsd.org/en/articles/gjournal-desktop/
  - densone 5y ago
    My issue is performance. But I’ve only read about UFS performance. So it might be fine?
    - toast0 5y ago
      I don't think there's much (anything?) in UFS that would lead to poor performance other than the usual suspects: If your disk is slow or dieing, you might blame UFS, but it's not really UFS. I've had some vague issues with the I/O scheduler, which isn't really UFS, but at the same time, UFS may be the only real client of the I/O scheduler, I think ZFS does it's own thing, anyway the systems were UFS only. This is super vague, and I don't have more details, but I just want to put it out there. For one class of machines that had a lot of disks (about 12 ssds), did a pretty even mix of reads and writes, evenly spread across the disks, upgrading from FreeBSD X to X + 1 wasn't possible because there was a large performance reversion. I think this was 10 -> 11, but it's possible it was 11 -> 12. Because this came up while my work was in progress migrating to our acquirer's datacenter which included switching to their inhouse Linux distro, it made sense to just leave those hosts on the older OS, and not spend the time debugging this. We didn't have a way to test this without production load, but that had user impact, and it would take a while to show up. It's quite possible this was just a simple tuning error, or possibly a bug that has been fixed for some time; the symptoms were obvious though: processes waiting on io, but the disks had a lot of idle time. If you have a lot of files in a given directory, that's kind of slow, and IIRC, if the directory every had a lot of files, the extra space won't get reclaimed until the directory is deleted, even if most of the files are unlinked. (This isn't uncommon for filesystems, some filesystems handle it better than others, there are application level strategies to deal with hashing/deeper directory trees) If the filesystem ever gets too full, the strategy to search for free space changes to one that's less fast; it won't change back, so don't fill your disk too much. (This ends up being a good idea for SSD disk health too, and again isn't super unusual in filesystems, but some filesystems probably do better). tunefs(8) says: > The file system's ability to avoid fragmentation will be reduced when the total free space, including the reserve, drops below 15%. As free space approaches zero, throughput can degrade by up to a factor of three over the performance obtained at a 10% threshold. UFS has snapshots, which is great, but everyonce in a while, you end up with a snapshot you forgot about, and it can really eat disk space and you may miss it. Not really a performance issue, but can lead to overfilling your drive. Of course, there's the obvious that UFS has no support for checksumming, but that's not performamce. Soft updates do allow for some amount of consistency in meta data, and background fsck is nice (but could tank performance, I suppose).
Bayart 5y ago
>There is controversy about Docker not running on FreeBSD but I believe (like many others) that FreeBSD has a more powerful tool. Jails are older and more mature - and by far - than any containerization solution on Linux. If FreeBSD jails and Solaris zones were equivalent to Linux containers, we'd have seen them take over the backend already. We haven't. They're really useful, they provided a degree of safety and peace of mind for multi-tenancy but they're not granular enough for what's done with $CONTAINER_RUNTIME these days. Jérôme Petazzoni has an old talk where he touches upon container primitives and compared them to jails : https://www.youtube.com/watch?v=sK5i-N34im8 https://www.youtube.com/watch?v=sK5i-N34im8
- yjftsjthsd-h 5y ago
  I think the problem is that docker is an excellent frontend, and zones and jails are excellent backends. People who say jails are better are probably right but they're missing the point, because they're not really solving the same problem; until I can use jails to create a container image, push it to a registry, and pull it from that registry and run it on a dozen servers - and do each of those steps in a single trivial command - jails are not useful for the thing that people care about docker for.
- area51org 5y ago
  Jails are not a replacement for containers.
frankjr 5y ago
> FreeBSD's network stack is (still) superior to Linux's - and, often, so is its performance. Where is this coming from exactly? The linked article about Facebook is 7 years old. The following benchmark shows the exact opposite: Linux's network stack has long surpassed FreeBSD's. And I would expect nothing else given the amount of work that has gone into Linux compared to FreeBSD. https://matteocroce.medium.com/linux-and-freebsd-networking-cbadcdb15ddd https://matteocroce.medium.com/linux-and-freebsd-networking-...
- drewg123 5y ago
  It depends on your workload. For static content, especially kTLS encrypted static content, FreeBSD is quite a bit better.
  - frankjr 5y ago
    Do you have any numbers you can share publicly? What's the reason it's better (Linux has in-kernel TLS as well, correct?)?. It would make for a great topic for Netflix TechBlog.
    - doublerabbit 5y ago
      > At this point, we’re able to serve 100% TLS traffic comfortably at 90 Gbps using the default FreeBSD TCP stack. https://netflixtechblog.com/serving-100-gbps-from-an-open-connect-appliance-cdb51dda3b99?source=---------0----------------------- https://netflixtechblog.com/serving-100-gbps-from-an-open-co...
      - loeg 5y ago
        That was 2017 -- it's quite a bit higher now.
    - drewg123 5y ago
      Comparing to Linux? No. There are no public numbers. However, I'm up to serving 709Gb/s of TLS encrypted traffic to from a single host to real Netflix customers with FreeBSD.
kodah 5y ago
> Linux has Docker, Podman, lxc, lxd, etc. but... FreeBSD has jails! Docker, podman, lxc, lxd, etc are userland components. Linux has cgroups and namespaces. FreeBSD jails are a bit more complicated because FreeBSD isn't distributed the way Linux is. Linux is distributed as just the kernel, whereas FreeBSD is a base OS. This probably could've been phrased better as, "Linux has no interest in userland and I want some userland consistency". That's fair, Linux was built around the idea that operating system diversity was a good thing long term, FreeBSD was more interested in consistency. I'm reading between the lines, a bit, here because of the critique of SystemD (note: not all linuxes use SystemD) Personally speaking, I like both Linuxes and FreeBSD but I don't think debating the two is important. Rather, I'd encourage turning your attention to the fact that every other component on a system runs an OS-like interface that we don't make open OS's or "firmware" for.
deleted 5y ago
[deleted]
embik 5y ago
The characterisation of systemd in this post really bothers me, particularly this: > 70 binaries just for initialising and logging It’s just not true. Those 70 binaries provide much more functionality than an init system, they can cover a significant portion of system management, including a local DNS resolver, network configuration or system time management. You can dislike the fact everything is so tightly integrated (which feels ironic given that the post goes on to praise a user space from one team), but let’s at least be correct about this.
- deleted 5y ago
  [deleted]
- hedora 5y ago
  > including a local DNS resolver, network configuration or system time management. Do. Not. Want. So, I have this pile of 70 binaries that are inexplicably tied to my init system, and (in my, informed enough for me, opinion) they're all garbage. How do I remove them without breaking init? This month's fresh hell: I have a problem where the systemd replacement for xscreensaver (logind, maybe? Good luck finding the culprit, let alone the manual!) won't accept my password unless I exit the current X session with "switch user" then restore the session using the normal login screen. There's a whole section on JWZ's xscreensaver page (from over a decade ago) explaining how to avoid this class of bug, but what does he know?!? That reminds me; I wonder if a *BSD is a good enough daily driver for the pine book pro yet (it's probably easier to port their kernel than to fix Linux userspace, after all...)
  - kimixa 5y ago
    > So, I have this pile of 70 binaries that are inexplicably tied to my init system, and (in my, informed enough for me, opinion) they're all garbage. How do I remove them without breaking init? Depending on distro, all can be substituted for alternatives (assuming they're used at all, I've seen a number of distros package the kitchen sink "just in case") Nothing about using systemd as pid1 required using any of their other developed tools, even logind. Find a distro that doesn't use them, or roll one yourself. It's a mistake to see "Systemd" as a single application - it's a collection of developed-together (so tend to work slightly nicer together) userspace tools - like FreeBSD without the libc and kernel, or GNU coreutils or similar. People don't seem to complain too much about those "Bundling Everything Together".
ppg677 5y ago
I could have sworn I read the same exact thing in 1997.
- m-i-l 5y ago
  Not sure about 1997, but definitely in 1998: I've been a Linux user since 1995, when I worked for a (then small now big) software company. I setup up their web site on Slackware Linux, building an early content management system, early content delivery network etc. But after I left in 1998, one of the first things my replacements did was change all the web servers to run NetBSD rather than Linux, because they said it had a better networking stack.
  - xianwen 5y ago
    Did you hear what happened to the NetBSD servers afterwards?
Melatonic 5y ago
I have been forced to work far too much with Citrix Netscaler virtual networking appliances and while I can see how it was probably a great product before Citrix purchased it the amount of bugs and regular security holes in it is insane. Especially for a damn networking appliance! That being said it also forced me to use FreeBSD a lot more than I ever would have otherwise and I have a lot of respect for the OS itself. I would not use it everywhere but it has amazing latency which makes it obviously great for networking.
gorgoiler 5y ago
Things I actually care about: kernel that supports my hardware, ZFS for secure snapshotted data, scriptable tools to manage NICs and ppp and VPNs, a fast optimised C++ compiler, the latest versions of dynamic language runtimes, a shell, a text editor, a terminal multiplexer, a nerdy window manager and an evergreen browser. On that playing field, the integrated nature of FreeBSD is nice but it’s an asterisk on top of the kernel rather than anything approaching what makes up a the system part of an Operating System. Almost everything else comes from a third party (and I’m fine with that.) I haven’t used FreeBSD as a daily OS for over a decade though. What’s the new coolness?
oneplane 5y ago
While I get the author's reasoning, it makes me wonder at what scale, portability and level of automation and disposability all of this is done. Even if an OS is 'better', a VM with a short lifetime will generally be 'good enough' very quickly. If you add a very large ecosystem and lots of support (both open source and community as well as commercial support) and existing knowledge, FreeBSD doesn't immediately come to mind as a great option. If I were to go for an 'appliance' style system, that's where I would likely consider FreeBSD at some point, especially with ZFS snapshots and (for me) the reliably and fast BTX loader. Pumping out BSD images isn't hard (great distro tools!) and complete system updates (due to the mentioned "one team does the whole release") are a breeze as well. This is of course something we can do with systemd and things like debootstrap too, but from a OS-image-as-deployable perspective this will do just fine.
quags 5y ago
I use freebsd for one project on node/js and for ssh jumper boxes. I have also been admining linux boxes since 99. I have no hate for systemd - there just was a learning curve. I like a basic rc.conf set up that freebsd has. Everything can go in this one file for startups. Binary updates have been around for years so doing security updates are easy with no need to rebuild world or compile. You can use pkg for third party installs (binaries) although they don't always follow the version in ports. Security wise kern_securelevel / ugidfw for security. freebsd update also allows for easy updating in major os releases. ZFS on root just works on freebsd. PF / ipfw to me makes much more sense than iptables (I haven'ted really moved to nftables). When I compare to ubuntu which is the OS I use for linux mostly now: * kvm is superior to bhyve in every way * automating security updates via apt are better than a combination of freebsdupdate/pkg updates. Plus the deb packages are made by Ubuntu and just work. ports/pkgs are third party on freebsd * rebootless kernel updates exist for ubuntu * It is easier to find people familiar with linux right away Really though the learning curve of freebsd <-> linux is not high.
nix23 5y ago
Congratulation, i did the same ~5years ago, and cant be any happier, jails bhyve dtrace zfs/ufs pf geom-compressed pkg/ports etcetc...nearly every day i find some useful features, and when try them out...they work!!
acdha 5y ago
> Consider systemd - was there really a need for such a system? While it brought some advantages, it added some complexity to an otherwise extremely simple and functional system. It remains divisive to this day, with many asking, "but was it really necessary? Did the advantages it brought balance the disadvantages?" This is really telling for the level of analysis done: systemd has been the target from a small number of vocal complainers but most working sysadmins only notice it in that they routinely deal with tasks which are now a couple of systemd stanzas instead of having to cobble together some combination of shell scripts and third-party utilities. Confusing noise with numbers is a dangerous mistake here because almost nobody sits around randomly saying “this works well”.
smlacy 5y ago
Bikeshedding at it's finest!
redm 5y ago
I used FreeBSD for many years (on servers) between 2001-2009. I also used it as a personal machine in the 90's. We used it for stability, at which it did well.The real problem was that everything was moving to Linux. The Linux kernel and community kept up with with bleeding edge hardware or software. Stability of Linux continued to improve and most people stopped compiling custom kernels anyway. I used to compile most user-space software too, and almost never do now. That largely negated the FreeBSD benefits.
- TedShiller 5y ago
  Interesting, I had the exact opposite experience. Because Linux always felt the need to support the latest gadgets, it became less stable over time. There is no feature in Linux that I use now that I did not use 10 years ago.
area51org 5y ago
All the "advantages" of FreeBSD are really just personal preferences, and little more. E.g., FreeBSD jails are not a replacement for containerization in any way. The FreeBSD network stack is better? I'll bet you can talk to a Linux kernel expert who will explain why exactly the opposite is true. And things being "simpler" in *BSD? Simpler is not always better. SystemD may be somewhat over-engineered, but it's also powerful as hell and can do things the old rc.X system couldn't dream of doing. There's nothing wrong with switching to another OS, but implying it's because the other OS is somehow empirically "better" is misguided.
znpy 5y ago
It's 2022 and if you still can't see the good in systemd then it's you choosing ignorance. Related: https://www.youtube.com/watch?v=o_AIw9bGogo https://www.youtube.com/watch?v=o_AIw9bGogo -- The tragedy of systemd. Where Benno Rice (FreeBSD Committer / FreeBSD Core member) explains the value of something like systemd.
29athrowaway 5y ago
Also remember that Darwin, the kernel used in macOS, is in part derived from FreeBSD.
markstos 5y ago
I ran FreeBSD servers for about a decade. Now all my servers are Linux with systemd. I'm liked FreeBSD then, I'm happy with systemd now. I have commits in both. I'm glad there are some people who use and prefer FreeBSD and other init system now, because diversity in digital ecosystems is benefits the whole just as diversity in natural ecosystems do. The shot taking at systemd here was disingenuous though. The author complained about the number of different systemd binaries and the lines of source code, but all these tools provide a highly consistent "system layer" with standardized conventions and high quality documentation-- it's essentially the same argument made to support FreeBSD as a large body of kernel and userspace code that's maintained in harmony.
Thaxll 5y ago
FreeBSD is most likely slower than linux in most scenarios. ZFS is supported natively in Linux ( Ubuntu ), jail are terrible compared to Docker, since Docker is very popular there are a millions tools built arround it, it's not just for sand boxing, it's a part of a complet development process. Who cares about boot process on a server seriously? "FreeBSD's network stack is (still) superior to Linux's - and, often, so is its performance." This is wrong, if it was the case most large compagnies would use BSD, atm they all use Linux, the only large compagny using BSD is Netflix because they added some tls offloading in kernel for their CDN which could have been done in Linux btw. imo don't use tech that is not widely used, you're going to reinvent the wheele in a worse way because tool a.b.c is missing.
- tester756 5y ago
  >imo don't use tech that is not widely used, you're going to reinvent the wheele in a worse way because tool a.b.c is missing. so kinda windows with wsl v2 is the way to go
- NexRebular 5y ago
  any source on all large companies using linux instead of e.g. Windows?
scrubs 5y ago
https://news.ycombinator.com/item?id=28584738 https://news.ycombinator.com/item?id=28584738 not Linux.
hakube 5y ago
Looking forward to your next article when you move your stuff back to Linux
gerdesj 5y ago
The key point is not Linux vs FreeBSD. It is simply choice. You have a real choice. Do it this way or that - do it your way. I like both Linux and FreeBSD but I deploy them differently. I slap Linux on my servers and desktops and I deploy FreeBSD via pfSense on firewalls. Sometimes I do experiments and try out BSD on the desktop which hasn't worked out yet for me but I live in hope because I adore *BSD as much as I do Linux. If BSD is the way to get your servers to do what you want then lovely. Do it and remember you have choice.

445 points by NexRebular 5y ago

tl;dr - FreeBSD has ample nice features for their use case, and is considerably simpler. Linux has loads of unneeded (for their use case) features, and so many cooks in the kitchen that the ongoing cognitive load (to keep track of the features and complexity and changes) looks worse than the one-time load of switching over to FreeBSD.

blakesterz 5y ago

"Some time ago we started a complex, continuous and not always linear operation, that is to migrate, where possible, most of the servers (ours and of our customers) from Linux to FreeBSD." I don't really disagree with any of the stated reasons, but I also didn't see a reason that would make me even consider making the move with our servers, or even bother with some small number of servers. At least for me, I'd need a bunch of REALLY GOOD reasons to consider a move like that. A huge cost savings AND some huge time savings in the future might do it.

johnklos 5y ago
Some people see the bigger picture and recognize that a medium amount of work now is better than lots of small amounts of work stretched over many years. Likewise, some people and many businesses see the immediate now and aren't always the best at planning for long term, and/or are overly optimistic that their pain points will eventually be fixed.
- pm90 5y ago
  > Likewise, some people and many businesses see the immediate now and aren't always the best at planning for long term, and/or are overly optimistic that their pain points will eventually be fixed. But that's the thing. The pain points mentioned in this article aren't really that strong to need to ditch the OS and move to a new one. These kinds of decisions have huge tradeoffs. One could argue, in fact, that moving to a non-traditional OS will make it much harder to hire experts or hand off the system to another team in the future.
  - djbusby 5y ago
    What? FreeBSD as a non-traditonal OS? I must disagree, FreeBSD may not be as common as Windows or Linux but it's not like Plan9 (from outer space) or BeOS.
  - washadjeffmad 5y ago
    I think of it this way: If you interpret "rolling stones gather no moss" to mean you've always got to keep pushing forward or you'll become obsolete, choose Linux. If you venerate moss as proof of the stability granted by doing the same thing simply and perfectly, you're probably already using FreeBSD. After CentOS Stable was cancelled, I migrated a number of our platforms to FreeBSD because it met our needs and I enjoyed working on it. No surprises, nothing breaking, and most importantly, no drama.
raverbashing 5y ago
I agree. Nothing there seems to deliver explicit customer value when switching to FreeBSD. How will switching help you deliver your service? Or is it just a "nice to have thing"?
- linksnapzz 5y ago
  Under what circumstances does the choice of backend OS ever deliver "explicit customer value", so much so that the customer would care about said choice?
  - lp0_on_fire 5y ago
    Normally the customer doesn't care. They will care, however, if something goes wrong during the migration or some unforeseen issue comes up later that degrades their experience.
  - brimble 5y ago
    It is definitely the case that underlying architecture, certainly all the way down to the OS, can be the difference between "I can create, test, and deploy this feature in a week, and it will be rock-solid" and "it'll take months and still fail on some edge cases—and fixing those is not remotely in our budget".
  - nine_k 5y ago
    I'd hazard to say that it delivers customer value via two avenues: (1) better SLAs, and (2) lower expenses. If your backend OS starts to run software more efficiently, cost less to host, has less maintenance downtime, has fewer security incidents, has fewer crashes, etc, having changed to it has produced customer value.
  - Twirrim 5y ago
    > Under what circumstances does the choice of backend OS ever deliver "explicit customer value", so much so that the customer would care about said choice? Is the service working, is it not? (are there show stopping OS issues?) Are the performance characteristics we need there, or not? Is the service secure or not? There's a whole lot of things right through that entire space where the choice of operating system can make quite a fundamental difference. As a general statement, I wouldn't want to make this kind of a migration without something bordering on a killer feature that wasn't possible otherwise, or some fundamental driver problems (I've replaced Linux with FreeBSD on border devices in the past, due to particular issues with Linux in ways I couldn't afford to have keep biting me)
toast0 5y ago
I agree that the stated reasons don't sound very compelling. Maybe in aggregate, but not individually. But they left out one of the bigger reasons, IMHO. FreeBSD doesn't tend to churn user (admin) facing interfaces. This saves you time, because you still use ifconfig to configure interfaces, and you still use netstat to look at network statistics, etc; so you don't have to learn a new tool to do the same thing but differently every couple of years. Sure, there's three firewalls, but they're the same three firewalls since forever.
- crazy_hombre 5y ago
  ifconfig/netstat was deprecated more than a decade ago, that's more than a couple years don't you think?
  - NoSorryCannot 5y ago
    Since it was just an example, I don't think refuting this particular item will nullify the opinion. The idea, I think, is that there are always more pieces in a state of deprecation and replacement at any given time in Linux land than in FreeBSD land.
    - phkahler 5y ago
      I think that's just due to the pace of development. The BSDs are resource constrained, so they have to pick and choose what to work on. That is both a good thing and a bad thing. Here the benefit is less churn. On the downside, they're just catching the Wayland train recently. On the up side, by catching it late they didn't suffer a lot of the growing pains.
  - comex 5y ago
    Deprecated on Linux. But I for one can’t consign them to the dustbin of my memory, because on my Mac, they are not deprecated, while the `ip` command that replaces them on Linux does not exist. With this part of macOS being derived from FreeBSD, I don’t know whether that makes FreeBSD a savior or a villain. Personally I blame all of the major Unix-derived operating systems (Linux, macOS, BSDs), as none of them show any interest in standardizing any APIs or commands invented this millennium. The subset that’s common to all of them is frozen in time, and is slowly being replaced by new bits that aren’t. From epoll/kqueue to eBPF, containers/jails to seccomp/pledge, DBus/XPC to init systems… from low-level network configuration (ifconfig, ip) to high-level network configuration (whatever that is this month).

betaby 5y ago

TL;DR - for no compelling reasons.

eatonphil 5y ago
I don't think that's fair. Maybe half of their reasons are more on the subjective side but half of them are actual technical choices like wanting ufs/zfs and jails.
- ianai 5y ago
  It’s even made easy to TLDR by their reasons being headlined in larger font and bold from the rest of the text. My question is how well moving their systems will shake out longer term.
- betaby 5y ago
  zfs on FreeBSD is from the same sources as in Linux. There is not a single word in the article why to chose jails. Article is 90s style rant, which is not a bad thing.
  - zinekeller 5y ago
    > zfs on FreeBSD is from the same sources as in Linux. You should know that due to license incompatibilities (CDDL and GPLv2), it's not really as smooth as the BSD integration (I wonder how Canonical avoids this issue). You know that OpenZFS's codebase is a monorepo (for the most part) but the in-kernel implementation is vastly different.
    - mnd999 5y ago
      AFAIK, Canonical are just hoping nobody sues them. I use Zfs as my boot file system on FreeBSD and Arch Linux and both work great. Linux was a bit of a faff to set up though, FreeBSD was easy.
      - quags 5y ago

AnonHP 5y ago

I have a tangential question on this part: > I sometimes experienced severe system slowdowns due to high I/O, even if the data to be processed was not read/write dependent. On FreeBSD this does not happen, and if something is blocking, it blocks THAT operation, not the rest of the system. I’ve seen this for a long time in Windows, where any prolonged I/O brings the entire system down to its knees. But it also seems to affect macOS (which is based on FreeBSD) as a system, though it’s not as bad as on Windows. Has Windows improved on this over the years? I’m unable to tell.

philliphaydon 5y ago
I have no idea how operating systems work, but at a wild guess. If you're running the OS off the same disk thats threashing the io, and, its queued too many read/write operations to handle the reads for the OS? And maybe FreeBSD is just so small it effectively caches itself into memory and doesn't need the IO so it appears to function still? Curious to know the reason.
- philliphaydon 5y ago
  I guess based on the downvotes I’m wrong but they also don’t know? Reason I want to know is when I tried running ubuntu off a spinning disk and wanted to move some games to the drive it would transfer about 1gb then lock up. Didn’t happen with an SSD.
  - loeg 5y ago
    Yeah, in general wild speculation (especially if it happens to be incorrect, as it is in this case) is discouraged on this forum.
bopbeepboop 5y ago
yehNonsense0 5y ago
It’s either the disk is the bottleneck or the GUI process taking a back seat to the OS prioritizing the “real work” requested. Computers get faster, we throw more at them. Physics is still the law of the land. Truh truh truh trade offfss; in computer eng; truh truh truh trade offs sorry Bowie

mbreese 5y ago

I ran a FreeBSD ZFS NFS server for a cluster for quite a while. I loved it. It was simple and stable. The thing that led me away from FreeBSD (aside from IT not being happy with an "alternative" OS), was that I needed a clustered filesystem. We outgrew the stage where I was comfortable with a single node and where upgrading storage meant a new JBOD. Are there any FreeBSD-centric answers to Ceph or Gluster or Lustre or BeeGFS?

arminiusreturns 5y ago
I haven't checked on it in a while but dragonfly has HAMMER2, the last docs https://gitweb.dragonflybsd.org/dragonfly.git/blob/57614c517203403e7fe4a34c370c7151ba52c539:/sys/vfs/hammer2/DESIGN https://gitweb.dragonflybsd.org/dragonfly.git/blob/57614c517... Might be a future alternative in the space.
- loeg 5y ago
  HAMMER2 is not yet clustered, as far as I know. They're still working on single-node functionality (I think). https://gitweb.dragonflybsd.org/dragonfly.git/blob_plain/HEAD:/sys/vfs/hammer2/DESIGN https://gitweb.dragonflybsd.org/dragonfly.git/blob_plain/HEA... was last updated in 2018, and at the time much of the clustering logic is described as "under development" or "not specced."
prakhunov 5y ago
This changed a couple of years ago but GlusterFS does run on FreeBSD now. It's not BSD centric but it works. https://www.freshports.org/net/glusterfs/ https://www.freshports.org/net/glusterfs/
mikehotel 5y ago
The handbook and wiki has info on FreeBSD Highly Available Storage: https://docs.freebsd.org/en/books/handbook/disks/#disks-hast https://docs.freebsd.org/en/books/handbook/disks/#disks-hast https://wiki.freebsd.org/HAST https://wiki.freebsd.org/HAST
diekhans 5y ago

johnklos 5y ago

This articulates most of my frustrations with the Linux world. Some of the distros are very good, but some of us who have work to do cringe at the thought of bringing up newer versions of an OS just to check all the things that've broken and changed needlessly.

briffle 5y ago
I hear people always complaining about 'needless' changes like systemd over init. But I sure like how my modern database servers reboot in less than 30 seconds, vs 5-12 minutes when they were running RHEL 5. (yes, some of that is because we moved from BIOS to UEFI)
- zinekeller 5y ago
  Hmm, why do I feel that a) the older servers verifies memory banks fully before boot (which takes up minutes) and b) the change to solid-state media has the biggest impact and not on SystemD. Can you confirm that at least a) is not the reason for slow boot times?
  - antoinealb 5y ago
    I know that anecdote is not data, but I had an Arch Linux laptop when the distribution moved from Init scripts (SysV init ?) to systemd, and the boot time was easily cut in three, going from 30s to 10s, on exactly the same laptop. Of course switching from HDD to SSD was a huge improvement, but don't discount what systemd's efficient boot parallelism was able to achieve.
    - trasz 5y ago
      It’s worth keeping in mind that the old Linux rc scripts where quite mediocre compared to their BSD counterparts. They didn’t even have a dependency mechanism. So, it’s fine to compare sysv Linux scripts to systemd, but please don’t extrapolate that to other systems.

Cloudef 5y ago

Isnt bsd's tcp stack single threaded?

rwaksmunski 5y ago
Netflix is pushing 400Gbit/s of TLS traffic per server with 60% CPU load. WhatsApp was doing millions of concurrent TCP connections per server. FreeBSD's networking has been multi-threaded for a long time now.
- hestefisk 5y ago
  Esp thanks to in-kernel TLS, which is fantastic.
- technofiend 5y ago
  As seen most recently here https://news.ycombinator.com/item?id=28584738 https://news.ycombinator.com/item?id=28584738
Beermotor 5y ago
It has been multi-threaded since at least the early 2000s and most of the work for handling scaling beyond 32 cores was completed in 2012. https://www.cl.cam.ac.uk/teaching/1516/ConcDisSys/2015-ConcurrentSystems-1B-L8-handout.pdf https://www.cl.cam.ac.uk/teaching/1516/ConcDisSys/2015-Concu...
- trasz 5y ago
  Well, to be honest a whole lot has been done to FreeBSD network stack scalability quite recently (14-CURRENT), eg introduction of epoch(9) and the routing nexthop patches.
hyperionplays 5y ago
It's multithreaded. I run FRR Routing punching 400gbit+ @ 75% CPU load on FreeBSD 12. On the same hardware, Debian fell over at 1.8gbit.

CyberRabbi 5y ago

To be fair all of these reasons come down to personal preference (sans the TCP performance claim). E.g. he prefers FreeBSD’s performance monitoring tools to Linux’s monitoring tools, or he prefers FreeBSD’s user land to Linux’s user land. That’s fine but it’s not very persuasive.

rwaksmunski 5y ago
vmstat -z gives me counters of kernel limit failures on FreeBSD. Very useful when debugging errors in very high performance environments. Anybody knows what the Linux equivalent is? Say I need to know how many times file descriptor/network socket/firewall connection state/accepted connection limits were reached?
- CyberRabbi 5y ago
  If one doesn’t exist out of the box you can relatively simply roll your own using bcc https://github.com/iovisor/bcc https://github.com/iovisor/bcc.

marcodiego 5y ago

Linux took many markets. The HPC, for example, has been 100% linux in TOP500 for a few years already. Monopoly by FLOSS is still monopoly. Healthy competition is good for users and forces options to improve, see LLVM vs GCC. To sum up: healthy FLOSS competition is welcome and needed.

pjmlp 5y ago
Agreed, if UNIX as concept is ever to evolve, it cannot be bound at UNIX === Linux that many now seem to consider.
- Shared404 5y ago
  I think it's somewhat to late for UNIX to evolve in general. There's too many decades of cruft and backwards compatibility built up. Afaict, most interesting new OS's being built right now are similar to UNIX, but very explicitly not.
  - trenchgun 5y ago
    GNU is not UNIX, though.
  - marcodiego 5y ago
    I don't know about UNIX per se, but consider Linux and MacOS progress in the last two decades. MacOS showed that it is possible for UNIX to be successful on the desktop. During the same period, Linux scaled from embedded computers and smartphones to supercomputers and servers. In terms of innovations, I'd bet MacOS has evolved too. Although "logical partitions"-like solutions were already known for some time, Linux made it widespread through containers; io_uring allows high throughput syscall-less zero-copy data transfer and futex2 allows to implement NT synchronization semantics that are very common in game development. All that ignoring just how much the desktop changed! The UNIX children are definitely not sitting still.
    - Shared404 5y ago

gtsop 5y ago

It feels like the title is wrong. Instead of saying "Linux is bad because I encountered X problem in production, which would have been prevented by BSD" the author goes on to list why BSD is better in general outside his specific use case. Nothing wrong with the comparison probably, but I got the impression the author just really wanted to do the migration and found some reasons to do so, without actually needing it. Nothing wrong with that as well. It's just the expectations set by the title that are off

eatonphil 5y ago
Neither the HN title nor the blog title is saying Linux is bad though. The title seems pretty in line with the article to me.
- gtsop 5y ago
  The blog titles says "why we are migrating...". I didn't get any of that. I got " why I like BSD, and thus I am migrating"
tomxor 5y ago
I had a similar feeling, throughout reading the post I wanted to know what the specific issues were that made BSD better suited, it's all been too abstracted. I think many of us (including me) have a tendency to try to quickly generalise our experiences, even when it's not appropriate - and when we go onto explain things to others without the original context it can sound too abstract or come across as evangelical. Either way, it loses meaning without real examples.

ianai 5y ago

“ The system is consistent - kernel and userland are created and managed by the same team” Their first reason is really saying a lot but with few words. For one, there’s no systemd. The init system is maintained alongside the entire rest of the system which adds a lot of consistency. The documentation for FreeBSD is also almost always accurate and standard. Etc etc I think you also largely don’t need a docker or etc in it since jails have been native to the OS for decades. I’d want to do some cross comparison first though before committing to that statement. Shouldn’t be lost that the licensing is also much friendlier to business uses. There’s afaik no equivalent to rhel, for that matter. This goes both ways though as how would you hire a FreeBSD admin based on their resume without a rhce-like FreeBSD certification program? Edit-I’ll posit that since FreeBSD is smaller an entity wishing to add features to the OS might face either less backlash or at least enjoy more visibility from the top developers of the OS. Linus, for instance, just has a larger list of entities vying for his attention on issues and commits.

idoubtit 5y ago

The reasons are, for a large part, not on the technical side. I was surprised, because this this a lot of work for little visible gain. Here are the reasons, slightly abbreviated: > The whole system is managed by the same team Mostly philosophical. > FreeBSD development is less driven by commercial interests. Mostly philosophical. > Linux has Docker but FreeBSD has jails! IMO, this comparison is a mistake. In the Linux world, systemd's nspawn is very similar to Jails. It's a glorified chroot, with security and resource management. All the systemd tools work seemlessly with nspawn machines (e.g. `systemctl status`). Containers à la Docker are a different thing. BTW, I thought the last sentence about security issues with Docker images was strange. If you care about unmaintained images, build them yourself. On the other side, the FreeBSD official documentation about Jails has a big warning that starts with "Important: the official Jails are a powerful tool, but they are not a security panacea." > Linux has no official support for zfs and such Fair point, though I've heard about production systems with zfs on Linux. > The FreeBSD boot procedure is better than grub. YMMV > FreeBSD's network is more performant. Is there some conclusive recent benchmark about this. The post uses a 2014 post about ipv6 at Facebook, which I think is far from definitive today. Especially more since it "forgot" to mention that Facebook intended to enhance the "Linux kernel network stack to rival or exceed that of FreeBSD." Did they succeed over these 8 years ? > Straightforward system performance analysis The point is not about the quality of the tools, but the way each distribution packages them. Seems very very low impact to me. > FreeBSD's Bhyve against Linux's KVM The author reluctantly admits that KVM is more mature.

LeonenTheDK 5y ago
I have essentially the same take. The sysadmin at my company prefers FreeBSD for all these reasons (as such that's what we're running), and he's engaged me a tonne about FreeBSD but all I see is an operating system that's just as good as the other mainstream server Linux distributions. Except now we've got a system that's more difficult to hire for. "Any competent admin can learn it easily" is something I've been told but how many will want to when they could easily go their whole career without encountering it again? I like your point about Docker vs Jails, I haven't seen it discussed like that before. I keep hearing Jails are more secure than anything else, I'll have to read more into it. As far as the networking goes, I haven't seen any recent benchmarks to substantiate those claims either. However, considering Netflix uses FreeBSD on their edge nodes and has put a lot of work into the upstream to improve networking (among other things), it wouldn't surprise me if it's technically superior to the Linux stack. Clearly though Linux's networking isn't an issue for most organizations. And regarding ZFS, ZFS on Linux and FreeBSD's ZFS implementation are now one and the same. It would be nice to see some of the big distributions(or even the Linux kernel) integrate it more directly. This is probably a solid point in favor of FreeBSD, but it's not like it doesn't work in Linux. I'm not a systems guy, so I'm probably out of the loop on this, but Proxmox is the only distribution I've seen with ZFS as a filesystem out of the box, but I don't know how much production use Proxmox sees. I only run it on my home server. All that to basically say, I like FreeBSD conceptually. I'm just still not convinced that it's doing enough things better to warrant using it over a common Linux distribution for general computing purposes.

acatton 5y ago

Funny enough, I decided to play with FreeBSD for personal projects in 2020. I gave up and I am reverting all my servers to Linux in 2022, for the opposite of the reasons mentioned in this article. * Lack of systemd. Managing services through shell scripts is outdated to me. It feels very hacky, there is no way to specify dependencies, and auto-restarts in case of crashes. Many FreeBSD devs praise launchd, well... systemd is a clone of launchd. * FreeBSD jail are sub-optimal compared to systemd-nspawn. There are tons of tools to create freebsd jails (manually, ezjail, bastillebsd, etc…) half of them are deprecated. At the end all of your jails end up on the same loopback interface, making it hard to firewall. I couldn't find a way to have one network interface per jail. With Linux, debootstrap + machinectl and you're good to go. * Lack of security modules (such as SELinux) -- Edit: I should have written "Lack of good security module" * nftables is way easier to grasp than pf, and as fast as pf, and has atomic reloads.

DarylZero 5y ago
- DarylZero 5y ago
  Haters be voting.
SpaceInvader 5y ago
Regarding jails - I do use separate loopback device per jail, plus pf with nat. No issues with firewalling.
- densone 5y ago
  Same .. I don’t use the separate loop back. Just firewall what I need to firewall .
fullstop 5y ago
I've grown to love systemd. It solves a lot of my problems with init scripts, particularly the ones involving environment / PATH at boot time. I've made init scripts for things before which work when they are invoked manually, but not at boot time because PATH was different. With systemd I am confident that if it works through systemctl it will work at boot. Maybe I am not tuning Linux appropriately, but I have been in situations where a Linux system is overwhelmed / overloaded and I am unable to ssh to it. I have never had that experience with FreeBSD -- somehow ssh is always quick and responsive even if the OS is out of memory. Most of the systems that I deal with are Linux, but I still have a few FreeBSD systems around and they are extraordinarily stable.

TurningCanadian 5y ago

"Btrfs is great in its intentions but still not as stable as it should be after all these years of development." may have been true years ago, but doesn't seem to be anymore.

csdvrx 5y ago
Give it another 10 years, and we may get a stable alternative to ZFS that will live in the kernel tree.
- dralley 5y ago
  And maybe it will be bcachefs :)
traceroute66 5y ago
How's the old RAID56 problem in BTRFS coming on ? ;-)
- TurningCanadian 5y ago
  Pick any given feature and a FS may or not support it well. RAID56 did have a major update, but still has the write hole and isn't recommended for production. I think the point still stands though. There has been lots of stabilization work done to BTRFS, and anyone using production-recommended features should consider the filesystem stable.

themerone 5y ago

The Wireguard debacle scared me off from FreeBSD. It seems they put too much trust in committers and don't have a solid enough review process.

loeg 5y ago
That Ars article is so distorted that it should best be seen as creative fiction.
- themerone 5y ago
  Do you have a link to the real story?
- danachow 5y ago
  Who mentioned anything about an Ars article? I read the whole debacle on mailing lists and Twitter. That was enough to form my own conclusions and I did not come away from that impressed with the FreeBSD development environs. It definitely flys in the face that somehow FreeBSD upholds technical excellence above business interests as is being claimed in the article.
  - kevans91 5y ago
    It was quite blown out of proportion even outside of the published articles.
Melatonic 5y ago
What was the debacle?
- themerone 5y ago
  FreeBSD 13 came very close to shipping with a WireGaurd implementation with many bugs an vulerabilites that were quickly identified by the creator of the WireGaurd prototocal shortly after learning about the update. Attempts were made to fix it, but they eventually decided to ship 13.0 without wiregaurd. It was very policitcal because the company that sponsored the development had already promised the feature to customers.

tombert 5y ago

I don't have enough experience with FreeBSD (outside of FreeNAS seven years ago), but I've never had any success getting it to run on a laptop. Every time I've tried installing it on a laptop I get issues with either the WiFi card not working, issues with the 3D accelerator card not working at all, or the lid-close go to sleep functionality not working. I've been using Linux since I was a teenager, so it's not like I am a stranger to fixing driver issues, but it seemed like no amount of Googling was good enough for me fix these problems (googling is much harder when you don't have functioning wifi). As a result I've always just stuck with Linux (or macOS semi-recently, which I suppose is kind of BSD?).

sandworm101 5y ago
>> lid-close go to sleep functionality I've read hundreds of post surrounding this issue. I've used laptops for decades but I don't think I have ever used this feature. It doesn't take even a second to hit a couple keys and sleep a laptop. The only feature lower on my priority list would be syncing the RGB keyboard to soundcloud. But that's just me. Evidently the lid-close-sleep thing is of vital import to millions of laptop users. It's one of those things where I just shake my head in bewilderment.
- trasz 5y ago
  I use it all the time; it boils down to a single line in sysctl.conf: hw.acpi.lid_switch_state=“S3”
- Melatonic 5y ago
  One of the features I always actually make sure to turn off - I am in the same boat and there are many times when I want to close the lid of my laptop but still have some stuff chugging along. Saves a lot of battery that way!
- tombert 5y ago
  I mean, sure, I'm an engineer, I'm sure I can figure out a workaround, but I like the feature of going to sleep on lid close. It's a feature I use, and I couldn't get it working in FreeBSD.

lazyant 5y ago

FreeBSD is a nicer, more logical Unix than Linux in general. Now as soon as you have a package or hardware that you want to use and it's not supported by FreeBSD let us know how that goes.

edgyquant 5y ago
It may be a more “logical unix” but most engineers (including me) don’t care about that. I started using Linux in the mid 2000s and had never heard of unix at the time and only discovered it by reading about Linux.

densone 5y ago

First off FreeBSD FTW. I use it everywhere over Linux now for the first time in 25 years and couldn’t be happier. My only wish is that BSD had a better non-CoW file system. Databases and Blockchains are already CoW so it does irk me slightly to use zfs for them. That being said, I’ve never had a problem because of it.

chalst 5y ago
If your complaint about UFS is the lack of journalling, you might be interested in https://docs.freebsd.org/en/articles/gjournal-desktop/ https://docs.freebsd.org/en/articles/gjournal-desktop/
- densone 5y ago
  My issue is performance. But I’ve only read about UFS performance. So it might be fine?
  - toast0 5y ago
    I don't think there's much (anything?) in UFS that would lead to poor performance other than the usual suspects: If your disk is slow or dieing, you might blame UFS, but it's not really UFS. I've had some vague issues with the I/O scheduler, which isn't really UFS, but at the same time, UFS may be the only real client of the I/O scheduler, I think ZFS does it's own thing, anyway the systems were UFS only. This is super vague, and I don't have more details, but I just want to put it out there. For one class of machines that had a lot of disks (about 12 ssds), did a pretty even mix of reads and writes, evenly spread across the disks, upgrading from FreeBSD X to X + 1 wasn't possible because there was a large performance reversion. I think this was 10 -> 11, but it's possible it was 11 -> 12. Because this came up while my work was in progress migrating to our acquirer's datacenter which included switching to their inhouse Linux distro, it made sense to just leave those hosts on the older OS, and not spend the time debugging this. We didn't have a way to test this without production load, but that had user impact, and it would take a while to show up. It's quite possible this was just a simple tuning error, or possibly a bug that has been fixed for some time; the symptoms were obvious though: processes waiting on io, but the disks had a lot of idle time. If you have a lot of files in a given directory, that's kind of slow, and IIRC, if the directory every had a lot of files, the extra space won't get reclaimed until the directory is deleted, even if most of the files are unlinked. (This isn't uncommon for filesystems, some filesystems handle it better than others, there are application level strategies to deal with hashing/deeper directory trees) If the filesystem ever gets too full, the strategy to search for free space changes to one that's less fast; it won't change back, so don't fill your disk too much. (This ends up being a good idea for SSD disk health too, and again isn't super unusual in filesystems, but some filesystems probably do better). tunefs(8) says: > The file system's ability to avoid fragmentation will be reduced when the total free space, including the reserve, drops below 15%. As free space approaches zero, throughput can degrade by up to a factor of three over the performance obtained at a 10% threshold. UFS has snapshots, which is great, but everyonce in a while, you end up with a snapshot you forgot about, and it can really eat disk space and you may miss it. Not really a performance issue, but can lead to overfilling your drive. Of course, there's the obvious that UFS has no support for checksumming, but that's not performamce. Soft updates do allow for some amount of consistency in meta data, and background fsck is nice (but could tank performance, I suppose).

Bayart 5y ago

>There is controversy about Docker not running on FreeBSD but I believe (like many others) that FreeBSD has a more powerful tool. Jails are older and more mature - and by far - than any containerization solution on Linux. If FreeBSD jails and Solaris zones were equivalent to Linux containers, we'd have seen them take over the backend already. We haven't. They're really useful, they provided a degree of safety and peace of mind for multi-tenancy but they're not granular enough for what's done with $CONTAINER_RUNTIME these days. Jérôme Petazzoni has an old talk where he touches upon container primitives and compared them to jails : https://www.youtube.com/watch?v=sK5i-N34im8 https://www.youtube.com/watch?v=sK5i-N34im8

yjftsjthsd-h 5y ago
I think the problem is that docker is an excellent frontend, and zones and jails are excellent backends. People who say jails are better are probably right but they're missing the point, because they're not really solving the same problem; until I can use jails to create a container image, push it to a registry, and pull it from that registry and run it on a dozen servers - and do each of those steps in a single trivial command - jails are not useful for the thing that people care about docker for.
area51org 5y ago
Jails are not a replacement for containers.

frankjr 5y ago

> FreeBSD's network stack is (still) superior to Linux's - and, often, so is its performance. Where is this coming from exactly? The linked article about Facebook is 7 years old. The following benchmark shows the exact opposite: Linux's network stack has long surpassed FreeBSD's. And I would expect nothing else given the amount of work that has gone into Linux compared to FreeBSD. https://matteocroce.medium.com/linux-and-freebsd-networking-cbadcdb15ddd https://matteocroce.medium.com/linux-and-freebsd-networking-...

drewg123 5y ago
It depends on your workload. For static content, especially kTLS encrypted static content, FreeBSD is quite a bit better.
- frankjr 5y ago
  Do you have any numbers you can share publicly? What's the reason it's better (Linux has in-kernel TLS as well, correct?)?. It would make for a great topic for Netflix TechBlog.
  - doublerabbit 5y ago
    > At this point, we’re able to serve 100% TLS traffic comfortably at 90 Gbps using the default FreeBSD TCP stack. https://netflixtechblog.com/serving-100-gbps-from-an-open-connect-appliance-cdb51dda3b99?source=---------0----------------------- https://netflixtechblog.com/serving-100-gbps-from-an-open-co...
    - loeg 5y ago
      That was 2017 -- it's quite a bit higher now.
  - drewg123 5y ago
    Comparing to Linux? No. There are no public numbers. However, I'm up to serving 709Gb/s of TLS encrypted traffic to from a single host to real Netflix customers with FreeBSD.

kodah 5y ago

> Linux has Docker, Podman, lxc, lxd, etc. but... FreeBSD has jails! Docker, podman, lxc, lxd, etc are userland components. Linux has cgroups and namespaces. FreeBSD jails are a bit more complicated because FreeBSD isn't distributed the way Linux is. Linux is distributed as just the kernel, whereas FreeBSD is a base OS. This probably could've been phrased better as, "Linux has no interest in userland and I want some userland consistency". That's fair, Linux was built around the idea that operating system diversity was a good thing long term, FreeBSD was more interested in consistency. I'm reading between the lines, a bit, here because of the critique of SystemD (note: not all linuxes use SystemD) Personally speaking, I like both Linuxes and FreeBSD but I don't think debating the two is important. Rather, I'd encourage turning your attention to the fact that every other component on a system runs an OS-like interface that we don't make open OS's or "firmware" for.

deleted 5y ago

[deleted]

embik 5y ago

The characterisation of systemd in this post really bothers me, particularly this: > 70 binaries just for initialising and logging It’s just not true. Those 70 binaries provide much more functionality than an init system, they can cover a significant portion of system management, including a local DNS resolver, network configuration or system time management. You can dislike the fact everything is so tightly integrated (which feels ironic given that the post goes on to praise a user space from one team), but let’s at least be correct about this.

deleted 5y ago
[deleted]
hedora 5y ago
> including a local DNS resolver, network configuration or system time management. Do. Not. Want. So, I have this pile of 70 binaries that are inexplicably tied to my init system, and (in my, informed enough for me, opinion) they're all garbage. How do I remove them without breaking init? This month's fresh hell: I have a problem where the systemd replacement for xscreensaver (logind, maybe? Good luck finding the culprit, let alone the manual!) won't accept my password unless I exit the current X session with "switch user" then restore the session using the normal login screen. There's a whole section on JWZ's xscreensaver page (from over a decade ago) explaining how to avoid this class of bug, but what does he know?!? That reminds me; I wonder if a *BSD is a good enough daily driver for the pine book pro yet (it's probably easier to port their kernel than to fix Linux userspace, after all...)
- kimixa 5y ago
  > So, I have this pile of 70 binaries that are inexplicably tied to my init system, and (in my, informed enough for me, opinion) they're all garbage. How do I remove them without breaking init? Depending on distro, all can be substituted for alternatives (assuming they're used at all, I've seen a number of distros package the kitchen sink "just in case") Nothing about using systemd as pid1 required using any of their other developed tools, even logind. Find a distro that doesn't use them, or roll one yourself. It's a mistake to see "Systemd" as a single application - it's a collection of developed-together (so tend to work slightly nicer together) userspace tools - like FreeBSD without the libc and kernel, or GNU coreutils or similar. People don't seem to complain too much about those "Bundling Everything Together".

ppg677 5y ago

I could have sworn I read the same exact thing in 1997.

m-i-l 5y ago
Not sure about 1997, but definitely in 1998: I've been a Linux user since 1995, when I worked for a (then small now big) software company. I setup up their web site on Slackware Linux, building an early content management system, early content delivery network etc. But after I left in 1998, one of the first things my replacements did was change all the web servers to run NetBSD rather than Linux, because they said it had a better networking stack.
- xianwen 5y ago
  Did you hear what happened to the NetBSD servers afterwards?

Melatonic 5y ago

I have been forced to work far too much with Citrix Netscaler virtual networking appliances and while I can see how it was probably a great product before Citrix purchased it the amount of bugs and regular security holes in it is insane. Especially for a damn networking appliance! That being said it also forced me to use FreeBSD a lot more than I ever would have otherwise and I have a lot of respect for the OS itself. I would not use it everywhere but it has amazing latency which makes it obviously great for networking.

gorgoiler 5y ago

Things I actually care about: kernel that supports my hardware, ZFS for secure snapshotted data, scriptable tools to manage NICs and ppp and VPNs, a fast optimised C++ compiler, the latest versions of dynamic language runtimes, a shell, a text editor, a terminal multiplexer, a nerdy window manager and an evergreen browser. On that playing field, the integrated nature of FreeBSD is nice but it’s an asterisk on top of the kernel rather than anything approaching what makes up a the system part of an Operating System. Almost everything else comes from a third party (and I’m fine with that.) I haven’t used FreeBSD as a daily OS for over a decade though. What’s the new coolness?

oneplane 5y ago

While I get the author's reasoning, it makes me wonder at what scale, portability and level of automation and disposability all of this is done. Even if an OS is 'better', a VM with a short lifetime will generally be 'good enough' very quickly. If you add a very large ecosystem and lots of support (both open source and community as well as commercial support) and existing knowledge, FreeBSD doesn't immediately come to mind as a great option. If I were to go for an 'appliance' style system, that's where I would likely consider FreeBSD at some point, especially with ZFS snapshots and (for me) the reliably and fast BTX loader. Pumping out BSD images isn't hard (great distro tools!) and complete system updates (due to the mentioned "one team does the whole release") are a breeze as well. This is of course something we can do with systemd and things like debootstrap too, but from a OS-image-as-deployable perspective this will do just fine.

quags 5y ago

I use freebsd for one project on node/js and for ssh jumper boxes. I have also been admining linux boxes since 99. I have no hate for systemd - there just was a learning curve. I like a basic rc.conf set up that freebsd has. Everything can go in this one file for startups. Binary updates have been around for years so doing security updates are easy with no need to rebuild world or compile. You can use pkg for third party installs (binaries) although they don't always follow the version in ports. Security wise kern_securelevel / ugidfw for security. freebsd update also allows for easy updating in major os releases. ZFS on root just works on freebsd. PF / ipfw to me makes much more sense than iptables (I haven'ted really moved to nftables). When I compare to ubuntu which is the OS I use for linux mostly now: * kvm is superior to bhyve in every way * automating security updates via apt are better than a combination of freebsdupdate/pkg updates. Plus the deb packages are made by Ubuntu and just work. ports/pkgs are third party on freebsd * rebootless kernel updates exist for ubuntu * It is easier to find people familiar with linux right away Really though the learning curve of freebsd <-> linux is not high.

nix23 5y ago

Congratulation, i did the same ~5years ago, and cant be any happier, jails bhyve dtrace zfs/ufs pf geom-compressed pkg/ports etcetc...nearly every day i find some useful features, and when try them out...they work!!

acdha 5y ago

> Consider systemd - was there really a need for such a system? While it brought some advantages, it added some complexity to an otherwise extremely simple and functional system. It remains divisive to this day, with many asking, "but was it really necessary? Did the advantages it brought balance the disadvantages?" This is really telling for the level of analysis done: systemd has been the target from a small number of vocal complainers but most working sysadmins only notice it in that they routinely deal with tasks which are now a couple of systemd stanzas instead of having to cobble together some combination of shell scripts and third-party utilities. Confusing noise with numbers is a dangerous mistake here because almost nobody sits around randomly saying “this works well”.

smlacy 5y ago

Bikeshedding at it's finest!

redm 5y ago

I used FreeBSD for many years (on servers) between 2001-2009. I also used it as a personal machine in the 90's. We used it for stability, at which it did well.The real problem was that everything was moving to Linux. The Linux kernel and community kept up with with bleeding edge hardware or software. Stability of Linux continued to improve and most people stopped compiling custom kernels anyway. I used to compile most user-space software too, and almost never do now. That largely negated the FreeBSD benefits.

TedShiller 5y ago
Interesting, I had the exact opposite experience. Because Linux always felt the need to support the latest gadgets, it became less stable over time. There is no feature in Linux that I use now that I did not use 10 years ago.

area51org 5y ago

All the "advantages" of FreeBSD are really just personal preferences, and little more. E.g., FreeBSD jails are not a replacement for containerization in any way. The FreeBSD network stack is better? I'll bet you can talk to a Linux kernel expert who will explain why exactly the opposite is true. And things being "simpler" in *BSD? Simpler is not always better. SystemD may be somewhat over-engineered, but it's also powerful as hell and can do things the old rc.X system couldn't dream of doing. There's nothing wrong with switching to another OS, but implying it's because the other OS is somehow empirically "better" is misguided.

znpy 5y ago

It's 2022 and if you still can't see the good in systemd then it's you choosing ignorance. Related: https://www.youtube.com/watch?v=o_AIw9bGogo https://www.youtube.com/watch?v=o_AIw9bGogo -- The tragedy of systemd. Where Benno Rice (FreeBSD Committer / FreeBSD Core member) explains the value of something like systemd.

29athrowaway 5y ago

Also remember that Darwin, the kernel used in macOS, is in part derived from FreeBSD.

markstos 5y ago

I ran FreeBSD servers for about a decade. Now all my servers are Linux with systemd. I'm liked FreeBSD then, I'm happy with systemd now. I have commits in both. I'm glad there are some people who use and prefer FreeBSD and other init system now, because diversity in digital ecosystems is benefits the whole just as diversity in natural ecosystems do. The shot taking at systemd here was disingenuous though. The author complained about the number of different systemd binaries and the lines of source code, but all these tools provide a highly consistent "system layer" with standardized conventions and high quality documentation-- it's essentially the same argument made to support FreeBSD as a large body of kernel and userspace code that's maintained in harmony.

Thaxll 5y ago

FreeBSD is most likely slower than linux in most scenarios. ZFS is supported natively in Linux ( Ubuntu ), jail are terrible compared to Docker, since Docker is very popular there are a millions tools built arround it, it's not just for sand boxing, it's a part of a complet development process. Who cares about boot process on a server seriously? "FreeBSD's network stack is (still) superior to Linux's - and, often, so is its performance." This is wrong, if it was the case most large compagnies would use BSD, atm they all use Linux, the only large compagny using BSD is Netflix because they added some tls offloading in kernel for their CDN which could have been done in Linux btw. imo don't use tech that is not widely used, you're going to reinvent the wheele in a worse way because tool a.b.c is missing.

tester756 5y ago
>imo don't use tech that is not widely used, you're going to reinvent the wheele in a worse way because tool a.b.c is missing. so kinda windows with wsl v2 is the way to go
NexRebular 5y ago
any source on all large companies using linux instead of e.g. Windows?

scrubs 5y ago

https://news.ycombinator.com/item?id=28584738 https://news.ycombinator.com/item?id=28584738 not Linux.

hakube 5y ago

Looking forward to your next article when you move your stuff back to Linux

gerdesj 5y ago

The key point is not Linux vs FreeBSD. It is simply choice. You have a real choice. Do it this way or that - do it your way. I like both Linux and FreeBSD but I deploy them differently. I slap Linux on my servers and desktops and I deploy FreeBSD via pfSense on firewalls. Sometimes I do experiments and try out BSD on the desktop which hasn't worked out yet for me but I live in hope because I adore *BSD as much as I do Linux. If BSD is the way to get your servers to do what you want then lovely. Do it and remember you have choice.