7 ms·

The reasons are, for a large part, not on the technical side. I was surprised, because this this a lot of work for little visible gain. Here are the reasons, sl

The reasons are, for a large part, not on the technical side. I was surprised, because this this a lot of work for little visible gain. Here are the reasons, slightly abbreviated: > The whole system is managed by the same team Mostly philosophical. > FreeBSD development is less driven by commercial interests. Mostly philosophical. > Linux has Docker but FreeBSD has jails! IMO, this comparison is a mistake. In the Linux world, systemd's nspawn is very similar to Jails. It's a glorified chroot, with security and resource management. All the systemd tools work seemlessly with nspawn machines (e.g. `systemctl status`). Containers à la Docker are a different thing. BTW, I thought the last sentence about security issues with Docker images was strange. If you care about unmaintained images, build them yourself. On the other side, the FreeBSD official documentation about Jails has a big warning that starts with "Important: the official Jails are a powerful tool, but they are not a security panacea." > Linux has no official support for zfs and such Fair point, though I've heard about production systems with zfs on Linux. > The FreeBSD boot procedure is better than grub. YMMV > FreeBSD's network is more performant. Is there some conclusive recent benchmark about this. The post uses a 2014 post about ipv6 at Facebook, which I think is far from definitive today. Especially more since it "forgot" to mention that Facebook intended to enhance the "Linux kernel network stack to rival or exceed that of FreeBSD." Did they succeed over these 8 years ? > Straightforward system performance analysis The point is not about the quality of the tools, but the way each distribution packages them. Seems very very low impact to me. > FreeBSD's Bhyve against Linux's KVM The author reluctantly admits that KVM is more mature.

LeonenTheDK 5y ago
I have essentially the same take. The sysadmin at my company prefers FreeBSD for all these reasons (as such that's what we're running), and he's engaged me a tonne about FreeBSD but all I see is an operating system that's just as good as the other mainstream server Linux distributions. Except now we've got a system that's more difficult to hire for. "Any competent admin can learn it easily" is something I've been told but how many will want to when they could easily go their whole career without encountering it again? I like your point about Docker vs Jails, I haven't seen it discussed like that before. I keep hearing Jails are more secure than anything else, I'll have to read more into it. As far as the networking goes, I haven't seen any recent benchmarks to substantiate those claims either. However, considering Netflix uses FreeBSD on their edge nodes and has put a lot of work into the upstream to improve networking (among other things), it wouldn't surprise me if it's technically superior to the Linux stack. Clearly though Linux's networking isn't an issue for most organizations. And regarding ZFS, ZFS on Linux and FreeBSD's ZFS implementation are now one and the same. It would be nice to see some of the big distributions(or even the Linux kernel) integrate it more directly. This is probably a solid point in favor of FreeBSD, but it's not like it doesn't work in Linux. I'm not a systems guy, so I'm probably out of the loop on this, but Proxmox is the only distribution I've seen with ZFS as a filesystem out of the box, but I don't know how much production use Proxmox sees. I only run it on my home server. All that to basically say, I like FreeBSD conceptually. I'm just still not convinced that it's doing enough things better to warrant using it over a common Linux distribution for general computing purposes.
- Melatonic 5y ago
  I have mostly only seen FreeBSD used for virtual networking devices so I think if that is your speciality (maybe someone who is a devops / networking role - is there a name for that?) you probably encounter it quite a bit. I do not think I would bother learning it or using it for much outside of networking just because it would make hiring competent people harder. Many times you also need to think about what may not be the very best solution from a purely technical perspective but also what is the best from both a people and technical perspective. One of the things that annoys me to no end ( and I am not sure if this is something specific to Netscalers or FreeBSD networking appliances in general ) is that the damn VM's are the only thing I run that does not properly report memory and CPU usage to VMware. From the hypervisor perspective they are constantly running at 100% CPU (their support has told me that the system reserves the full amount of CPU given but is not actually running that high - and when you actually connect to the box it self reports the number correctly). Not a huge deal but it annoys me to have that little red "X" in the GUI next to the VM name all the time. That being said I also have not seen any recent FreeBSD vs Linux benchmarks but I would imagine if a company as large as Netflix is already using it en masse then there would need to be not just parity but tangible benefits to swap it all out for some other linux distro. An org starting from scratch of course would be a different beast.
  - LeonenTheDK 5y ago
    Solid point about networking. No one really seems to dispute its presence there and rightly so it seems (again, I'm a programmer not an admin so my personal experience is limited). > Many times you also need to think about what may not be the very best solution from a purely technical perspective but also what is the best from both a people and technical perspective. This is huge when it comes to running the company's software imho. It's useless having the perfect solution if you can't get anyone to run it when "good enough" will have a plethora of folks ready and willing. Especially when it comes to managing the bus factor. When I asked the admin at my org about that, he explicitly said he didn't care about it (that's more an indictment of him than FreeBSD though). Overall the human factor is something I've been trying to embrace more lately when it comes to work. For personal stuff though I'll be as esoteric as I like. > the damn VM's are the only thing I run that does not properly report memory and CPU usage to VMware Well that's just plain frustrating.
area51org 5y ago
Exactly. Personal preference is all fine and good, and he's got a right to his own opinions, absolutely. But to imply that these are empirical differences between FreeBSD and Linux, well, that's nonsense.
boring_twenties 5y ago
The systemd-nspawn man page includes the following: > Like all other systemd-nspawn features, this is not a security feature and provides protection against accidental destructive operations only Doesn't seem very similar to jails to me.
mcronce 5y ago
> Especially more since it "forgot" to mention that Facebook intended to enhance the "Linux kernel network stack to rival or exceed that of FreeBSD." Did they succeed over these 8 years ? I don't know about Facebook-driven efforts specifically offhand, but I recall attending a talk at LinuxCon 2015 _specifically_ about optimizing the network stack. Given by a Red Hat employee, IIRC.
- Twirrim 5y ago
  Facebook has spent a lot of engineering resources on the kernel. Can't speak to their network stack, but if you want to see what they've contributed, they use emails under the @fb.com domain: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?qt=author&q=%40fb.com https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... From a quick glance through I see a lot of work on eBPF stuff, memory, I/O subsystem, cgroups, and btrfs.
throwawaylinux 5y ago
> Is there some conclusive recent benchmark about this. The post uses a 2014 post about ipv6 at Facebook, which I think is far from definitive today. Especially more since it "forgot" to mention that Facebook intended to enhance the "Linux kernel network stack to rival or exceed that of FreeBSD." Did they succeed over these 8 years ? I would not be surprised if there still were some cases where FreeBSD could do better. The network stack in particular has a vast amount of performance heuristics and tuning where you can just happen to hit the right or wrong side of some performance jump depending on your exact case, for example. But to go from there to the network stack is more performant is puzzling. There are certainly cases where the Linux network stack is more performant than FreeBSD -- search for linux vs freebsd network performance and you'll find benchmark tests and complaints and anecdotes around the place including freebsd forums and mailing lists where FreeBSD is slower, and yet FreeBSD proponents would (rightly) say these are not proof Linux's network stack is faster and there might be all sorts of reasons for the differences. So you can't have it both ways. Netflix uses FreeBSD for some things yes. Not sure if number of big internet corporations using an OS would be a favorable metric for FreeBSD either. Again you can't have it both ways. Netflix says FreeBSD is capable of performing well in that kind of environment, it does not say that it outperforms Linux in general or even in that environment. Any more than Google says the opposite.
deleted 5y ago
[deleted]

LeonenTheDK 5y ago

I have essentially the same take. The sysadmin at my company prefers FreeBSD for all these reasons (as such that's what we're running), and he's engaged me a tonne about FreeBSD but all I see is an operating system that's just as good as the other mainstream server Linux distributions. Except now we've got a system that's more difficult to hire for. "Any competent admin can learn it easily" is something I've been told but how many will want to when they could easily go their whole career without encountering it again? I like your point about Docker vs Jails, I haven't seen it discussed like that before. I keep hearing Jails are more secure than anything else, I'll have to read more into it. As far as the networking goes, I haven't seen any recent benchmarks to substantiate those claims either. However, considering Netflix uses FreeBSD on their edge nodes and has put a lot of work into the upstream to improve networking (among other things), it wouldn't surprise me if it's technically superior to the Linux stack. Clearly though Linux's networking isn't an issue for most organizations. And regarding ZFS, ZFS on Linux and FreeBSD's ZFS implementation are now one and the same. It would be nice to see some of the big distributions(or even the Linux kernel) integrate it more directly. This is probably a solid point in favor of FreeBSD, but it's not like it doesn't work in Linux. I'm not a systems guy, so I'm probably out of the loop on this, but Proxmox is the only distribution I've seen with ZFS as a filesystem out of the box, but I don't know how much production use Proxmox sees. I only run it on my home server. All that to basically say, I like FreeBSD conceptually. I'm just still not convinced that it's doing enough things better to warrant using it over a common Linux distribution for general computing purposes.

Melatonic 5y ago
I have mostly only seen FreeBSD used for virtual networking devices so I think if that is your speciality (maybe someone who is a devops / networking role - is there a name for that?) you probably encounter it quite a bit. I do not think I would bother learning it or using it for much outside of networking just because it would make hiring competent people harder. Many times you also need to think about what may not be the very best solution from a purely technical perspective but also what is the best from both a people and technical perspective. One of the things that annoys me to no end ( and I am not sure if this is something specific to Netscalers or FreeBSD networking appliances in general ) is that the damn VM's are the only thing I run that does not properly report memory and CPU usage to VMware. From the hypervisor perspective they are constantly running at 100% CPU (their support has told me that the system reserves the full amount of CPU given but is not actually running that high - and when you actually connect to the box it self reports the number correctly). Not a huge deal but it annoys me to have that little red "X" in the GUI next to the VM name all the time. That being said I also have not seen any recent FreeBSD vs Linux benchmarks but I would imagine if a company as large as Netflix is already using it en masse then there would need to be not just parity but tangible benefits to swap it all out for some other linux distro. An org starting from scratch of course would be a different beast.
- LeonenTheDK 5y ago
  Solid point about networking. No one really seems to dispute its presence there and rightly so it seems (again, I'm a programmer not an admin so my personal experience is limited). > Many times you also need to think about what may not be the very best solution from a purely technical perspective but also what is the best from both a people and technical perspective. This is huge when it comes to running the company's software imho. It's useless having the perfect solution if you can't get anyone to run it when "good enough" will have a plethora of folks ready and willing. Especially when it comes to managing the bus factor. When I asked the admin at my org about that, he explicitly said he didn't care about it (that's more an indictment of him than FreeBSD though). Overall the human factor is something I've been trying to embrace more lately when it comes to work. For personal stuff though I'll be as esoteric as I like. > the damn VM's are the only thing I run that does not properly report memory and CPU usage to VMware Well that's just plain frustrating.

area51org 5y ago

Exactly. Personal preference is all fine and good, and he's got a right to his own opinions, absolutely. But to imply that these are empirical differences between FreeBSD and Linux, well, that's nonsense.

boring_twenties 5y ago

The systemd-nspawn man page includes the following: > Like all other systemd-nspawn features, this is not a security feature and provides protection against accidental destructive operations only Doesn't seem very similar to jails to me.

mcronce 5y ago

> Especially more since it "forgot" to mention that Facebook intended to enhance the "Linux kernel network stack to rival or exceed that of FreeBSD." Did they succeed over these 8 years ? I don't know about Facebook-driven efforts specifically offhand, but I recall attending a talk at LinuxCon 2015 _specifically_ about optimizing the network stack. Given by a Red Hat employee, IIRC.

Twirrim 5y ago
Facebook has spent a lot of engineering resources on the kernel. Can't speak to their network stack, but if you want to see what they've contributed, they use emails under the @fb.com domain: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/?qt=author&q=%40fb.com https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... From a quick glance through I see a lot of work on eBPF stuff, memory, I/O subsystem, cgroups, and btrfs.

throwawaylinux 5y ago

> Is there some conclusive recent benchmark about this. The post uses a 2014 post about ipv6 at Facebook, which I think is far from definitive today. Especially more since it "forgot" to mention that Facebook intended to enhance the "Linux kernel network stack to rival or exceed that of FreeBSD." Did they succeed over these 8 years ? I would not be surprised if there still were some cases where FreeBSD could do better. The network stack in particular has a vast amount of performance heuristics and tuning where you can just happen to hit the right or wrong side of some performance jump depending on your exact case, for example. But to go from there to the network stack is more performant is puzzling. There are certainly cases where the Linux network stack is more performant than FreeBSD -- search for linux vs freebsd network performance and you'll find benchmark tests and complaints and anecdotes around the place including freebsd forums and mailing lists where FreeBSD is slower, and yet FreeBSD proponents would (rightly) say these are not proof Linux's network stack is faster and there might be all sorts of reasons for the differences. So you can't have it both ways. Netflix uses FreeBSD for some things yes. Not sure if number of big internet corporations using an OS would be a favorable metric for FreeBSD either. Again you can't have it both ways. Netflix says FreeBSD is capable of performing well in that kind of environment, it does not say that it outperforms Linux in general or even in that environment. Any more than Google says the opposite.

deleted 5y ago

[deleted]