6 ms·

I have essentially the same take. The sysadmin at my company prefers FreeBSD for all these reasons (as such that's what we're running), and he's engaged me a to

I have essentially the same take. The sysadmin at my company prefers FreeBSD for all these reasons (as such that's what we're running), and he's engaged me a tonne about FreeBSD but all I see is an operating system that's just as good as the other mainstream server Linux distributions. Except now we've got a system that's more difficult to hire for. "Any competent admin can learn it easily" is something I've been told but how many will want to when they could easily go their whole career without encountering it again? I like your point about Docker vs Jails, I haven't seen it discussed like that before. I keep hearing Jails are more secure than anything else, I'll have to read more into it. As far as the networking goes, I haven't seen any recent benchmarks to substantiate those claims either. However, considering Netflix uses FreeBSD on their edge nodes and has put a lot of work into the upstream to improve networking (among other things), it wouldn't surprise me if it's technically superior to the Linux stack. Clearly though Linux's networking isn't an issue for most organizations. And regarding ZFS, ZFS on Linux and FreeBSD's ZFS implementation are now one and the same. It would be nice to see some of the big distributions(or even the Linux kernel) integrate it more directly. This is probably a solid point in favor of FreeBSD, but it's not like it doesn't work in Linux. I'm not a systems guy, so I'm probably out of the loop on this, but Proxmox is the only distribution I've seen with ZFS as a filesystem out of the box, but I don't know how much production use Proxmox sees. I only run it on my home server. All that to basically say, I like FreeBSD conceptually. I'm just still not convinced that it's doing enough things better to warrant using it over a common Linux distribution for general computing purposes.

Melatonic 5y ago
I have mostly only seen FreeBSD used for virtual networking devices so I think if that is your speciality (maybe someone who is a devops / networking role - is there a name for that?) you probably encounter it quite a bit. I do not think I would bother learning it or using it for much outside of networking just because it would make hiring competent people harder. Many times you also need to think about what may not be the very best solution from a purely technical perspective but also what is the best from both a people and technical perspective. One of the things that annoys me to no end ( and I am not sure if this is something specific to Netscalers or FreeBSD networking appliances in general ) is that the damn VM's are the only thing I run that does not properly report memory and CPU usage to VMware. From the hypervisor perspective they are constantly running at 100% CPU (their support has told me that the system reserves the full amount of CPU given but is not actually running that high - and when you actually connect to the box it self reports the number correctly). Not a huge deal but it annoys me to have that little red "X" in the GUI next to the VM name all the time. That being said I also have not seen any recent FreeBSD vs Linux benchmarks but I would imagine if a company as large as Netflix is already using it en masse then there would need to be not just parity but tangible benefits to swap it all out for some other linux distro. An org starting from scratch of course would be a different beast.
- LeonenTheDK 5y ago
  Solid point about networking. No one really seems to dispute its presence there and rightly so it seems (again, I'm a programmer not an admin so my personal experience is limited). > Many times you also need to think about what may not be the very best solution from a purely technical perspective but also what is the best from both a people and technical perspective. This is huge when it comes to running the company's software imho. It's useless having the perfect solution if you can't get anyone to run it when "good enough" will have a plethora of folks ready and willing. Especially when it comes to managing the bus factor. When I asked the admin at my org about that, he explicitly said he didn't care about it (that's more an indictment of him than FreeBSD though). Overall the human factor is something I've been trying to embrace more lately when it comes to work. For personal stuff though I'll be as esoteric as I like. > the damn VM's are the only thing I run that does not properly report memory and CPU usage to VMware Well that's just plain frustrating.
- asveikau 5y ago
  I'm seeing a lot about how hiring competent people would be harder. But I think competence is independent of this, so perhaps what you're saying is Linux use masks incompetence? I would really question the chops of somebody who says they can work with Linux but not FreeBSD. People who really know their stuff on Linux would be able to figure it out. And if you never see FreeBSD again... So what? The stuff you are building on top is probably more relevant than questions like this.
  - HeckFeck 5y ago
    I've played with each of Linux, OSX and BSDs. The commands and syntax are similar enough, it would be a trifle, indeed a matter of pride for any *nix sysadmin to learn another Unix! And that much should be expected, since they are both in the same Unix family tree (even if Linux was "grafted in").
  - throwawaylinux 5y ago
    > perhaps what you're saying is Linux use masks incompetence? Why do I see so much of this snide needling from FreeBSD proponents? Small dog syndrome? It matters when you don't want to hire someone to learn a system, and someone who can land a job with their Linux skills does not necessarily want to learn a different system (one that's in a lot less demand). It's a valid concern even if not an insurmountable problem.
ashtonkem 5y ago
> However, considering Netflix uses FreeBSD on their edge nodes and has put a lot of work into the upstream to improve networking (among other things), it wouldn't surprise me if it's technically superior to the Linux stack. Possible. But it's also possible that the founder of the team was a BSD person. You occasionally get cases where the preferences of one key person affects things in the long run. At this point I'm sure that they're baked in, because removing all that code would not be worth the effort too, so even if it was better when Netflix went online, that doesn't guarantee that it's still true today. That being said, Linux is also used in a ton of places, including high network jobs. It would take a decent sized amount of evidence to convince me that all of those other places were wrong and basically only Netflix was right to use BSD in network heavy cases. My suspicion is that either: 1. The difference is minimal to negligent, and not enough to justify mixed OS development or 2. The difference is significant, but you have to be pushing your network much harder than most teams ever do for it to show up.
- ksec 5y ago
  Replying to all the post above as well. FreeBSD networking during WhatsApp and Netflix starting era was definitely better than Linux. Both in their specific usage pattern. So it wasn’t some ideological reason but actual technical one. But that was 10+ years ago. If I remember correctly from reading Meta / Facebook successfully switch all of WhatsApp large bare metal 4U server to their standardise Linux blade by 2017. They also spent 3 years working on Erlang BEAM and Networking so it was working close enough. Netflix continues to use FreeBSD and it is working well for them. Both the lead engineer comment and submit update on HN frequently. Do a search and it should be easy to find it. ( on my phone now so can’t quote links ) 2022 now I think any general advantage of FreeBSD over Linux in networking performance would likely be minimal. But it also doesn’t make sense to switch to another for the sake of OS unification.
  - ybloviator 5y ago
    If anyone wants to learn (as opposed to arguing), the papers on Netflix's TLS offloading work are a fun read. And say what you will, something like 20% of all internet traffic has a FreeBSD endpoint. And doing 400Gb/s of encrypted streaming from one box is quite an accomplishment. I would argue the reason someone like Netflix or any of the other large orgs using FreeBSD come there is for the simplicity/cohesiveness. If you're looking to do something like in-kernel TLS or something, way easier on something smaller, documented, and with an OS devel team that will likely incorporate your work in future releases.
zucker42 5y ago
> It would be nice to see some of the big distributions(or even the Linux kernel) integrate it more directly. What's holding this back is legal issues, since ZFS's licenses is not compatible with the GPLv2.
chungy 5y ago
> ZFS on Linux and FreeBSD's ZFS implementation are now one and the same. Don't assume that this means feature parity, not by a long shot. ZFS on FreeBSD affords you native NFSv4 ACLs (hugely important in corporate settings), tight integration with jails, much better delegation support, the FreeBSD boot loader is actually updated to understand all the ZFS features, and offers boot environments and checkpoint rollbacks right from the boot loader menu (and not having to shuffle around with a dedicated /boot partition like you do on Linux). If none of those features appeal to you, ZFS on Linux will seem very much the same. Indeed, within the limitations of Linux itself, ZFS does very well at providing every feature that Linux lets it provide.

Melatonic 5y ago

I have mostly only seen FreeBSD used for virtual networking devices so I think if that is your speciality (maybe someone who is a devops / networking role - is there a name for that?) you probably encounter it quite a bit. I do not think I would bother learning it or using it for much outside of networking just because it would make hiring competent people harder. Many times you also need to think about what may not be the very best solution from a purely technical perspective but also what is the best from both a people and technical perspective. One of the things that annoys me to no end ( and I am not sure if this is something specific to Netscalers or FreeBSD networking appliances in general ) is that the damn VM's are the only thing I run that does not properly report memory and CPU usage to VMware. From the hypervisor perspective they are constantly running at 100% CPU (their support has told me that the system reserves the full amount of CPU given but is not actually running that high - and when you actually connect to the box it self reports the number correctly). Not a huge deal but it annoys me to have that little red "X" in the GUI next to the VM name all the time. That being said I also have not seen any recent FreeBSD vs Linux benchmarks but I would imagine if a company as large as Netflix is already using it en masse then there would need to be not just parity but tangible benefits to swap it all out for some other linux distro. An org starting from scratch of course would be a different beast.

LeonenTheDK 5y ago
Solid point about networking. No one really seems to dispute its presence there and rightly so it seems (again, I'm a programmer not an admin so my personal experience is limited). > Many times you also need to think about what may not be the very best solution from a purely technical perspective but also what is the best from both a people and technical perspective. This is huge when it comes to running the company's software imho. It's useless having the perfect solution if you can't get anyone to run it when "good enough" will have a plethora of folks ready and willing. Especially when it comes to managing the bus factor. When I asked the admin at my org about that, he explicitly said he didn't care about it (that's more an indictment of him than FreeBSD though). Overall the human factor is something I've been trying to embrace more lately when it comes to work. For personal stuff though I'll be as esoteric as I like. > the damn VM's are the only thing I run that does not properly report memory and CPU usage to VMware Well that's just plain frustrating.
asveikau 5y ago
I'm seeing a lot about how hiring competent people would be harder. But I think competence is independent of this, so perhaps what you're saying is Linux use masks incompetence? I would really question the chops of somebody who says they can work with Linux but not FreeBSD. People who really know their stuff on Linux would be able to figure it out. And if you never see FreeBSD again... So what? The stuff you are building on top is probably more relevant than questions like this.
- HeckFeck 5y ago
  I've played with each of Linux, OSX and BSDs. The commands and syntax are similar enough, it would be a trifle, indeed a matter of pride for any *nix sysadmin to learn another Unix! And that much should be expected, since they are both in the same Unix family tree (even if Linux was "grafted in").
- throwawaylinux 5y ago
  > perhaps what you're saying is Linux use masks incompetence? Why do I see so much of this snide needling from FreeBSD proponents? Small dog syndrome? It matters when you don't want to hire someone to learn a system, and someone who can land a job with their Linux skills does not necessarily want to learn a different system (one that's in a lot less demand). It's a valid concern even if not an insurmountable problem.

ashtonkem 5y ago

> However, considering Netflix uses FreeBSD on their edge nodes and has put a lot of work into the upstream to improve networking (among other things), it wouldn't surprise me if it's technically superior to the Linux stack. Possible. But it's also possible that the founder of the team was a BSD person. You occasionally get cases where the preferences of one key person affects things in the long run. At this point I'm sure that they're baked in, because removing all that code would not be worth the effort too, so even if it was better when Netflix went online, that doesn't guarantee that it's still true today. That being said, Linux is also used in a ton of places, including high network jobs. It would take a decent sized amount of evidence to convince me that all of those other places were wrong and basically only Netflix was right to use BSD in network heavy cases. My suspicion is that either: 1. The difference is minimal to negligent, and not enough to justify mixed OS development or 2. The difference is significant, but you have to be pushing your network much harder than most teams ever do for it to show up.

ksec 5y ago
Replying to all the post above as well. FreeBSD networking during WhatsApp and Netflix starting era was definitely better than Linux. Both in their specific usage pattern. So it wasn’t some ideological reason but actual technical one. But that was 10+ years ago. If I remember correctly from reading Meta / Facebook successfully switch all of WhatsApp large bare metal 4U server to their standardise Linux blade by 2017. They also spent 3 years working on Erlang BEAM and Networking so it was working close enough. Netflix continues to use FreeBSD and it is working well for them. Both the lead engineer comment and submit update on HN frequently. Do a search and it should be easy to find it. ( on my phone now so can’t quote links ) 2022 now I think any general advantage of FreeBSD over Linux in networking performance would likely be minimal. But it also doesn’t make sense to switch to another for the sake of OS unification.
- ybloviator 5y ago
  If anyone wants to learn (as opposed to arguing), the papers on Netflix's TLS offloading work are a fun read. And say what you will, something like 20% of all internet traffic has a FreeBSD endpoint. And doing 400Gb/s of encrypted streaming from one box is quite an accomplishment. I would argue the reason someone like Netflix or any of the other large orgs using FreeBSD come there is for the simplicity/cohesiveness. If you're looking to do something like in-kernel TLS or something, way easier on something smaller, documented, and with an OS devel team that will likely incorporate your work in future releases.

zucker42 5y ago

> It would be nice to see some of the big distributions(or even the Linux kernel) integrate it more directly. What's holding this back is legal issues, since ZFS's licenses is not compatible with the GPLv2.

chungy 5y ago

> ZFS on Linux and FreeBSD's ZFS implementation are now one and the same. Don't assume that this means feature parity, not by a long shot. ZFS on FreeBSD affords you native NFSv4 ACLs (hugely important in corporate settings), tight integration with jails, much better delegation support, the FreeBSD boot loader is actually updated to understand all the ZFS features, and offers boot environments and checkpoint rollbacks right from the boot loader menu (and not having to shuffle around with a dedicated /boot partition like you do on Linux). If none of those features appeal to you, ZFS on Linux will seem very much the same. Indeed, within the limitations of Linux itself, ZFS does very well at providing every feature that Linux lets it provide.