5 ms·

Let me pick and choose a few points since replying to everything will necessitate a similar 40-page post. FreeBSD has great engineering and release managemen

Let me pick and choose a few points since replying to everything will necessitate a similar 40-page post. FreeBSD has great engineering and release management practices When someone gets an idea and develops something new, it first gets peer technical reviews The recent WireGuard debacle left a bad taste about this. As it actually turns out, sometimes there is zero technical review for very important patches and a few blessed developers can (and sometimes do) just throw their stuff directly into trunk. Unlike on Linux, the ZFS filesystem is a first class citizen on FreeBSD ZFS has first-class support on Ubuntu and is compiled into the kernel. FreeBSD also supports boot environments Including this one IIRC. Jails Well... linux has containers, and if all you need is isolation there's this: https://news.ycombinator.com/item?id=29976096 https://news.ycombinator.com/item?id=29976096 systemd is not required (all the heavy lifting is done by the kernel using the same features employed by containers), but it's available almost everywhere and makes this easy. security I actually think it's worse in this regard because of the link above. Most services on my machines are heavily locked down and isolated from each other since systemd makes this very easy (add a few key-value pairs to an .ini file and it's done). On FreeBSD the developer must add capsicum support (which is not easy to say the least), or you have to setup jails for each and every application manually. Capsicum Excluding the usual boring stuff like seccomp-bpf, we now also have this: https://lwn.net/Articles/859908 https://lwn.net/Articles/859908 It's pretty much unveil for Linux. I think it kind of flew under the radar. I don't believe there's anything like capcisum, though. DTrace https://www.brendangregg.com/dtrace.html#Linux https://www.brendangregg.com/dtrace.html#Linux There's also real DTrace on Oracle Linux if you're ready to sell your soul: https://docs.oracle.com/en/operating-systems/oracle-linux/dtrace-guide/dt_about.html#dt_prov_about https://docs.oracle.com/en/operating-systems/oracle-linux/dt... bhyve Not sure if this counts as a particular advantage. Firewall I find nftables to be pretty enjoyable to work with. It has a similar syntax, removes duplication of rules (supporting both ipv4 and ipv6 at the same time), etc. I actually removed firewalld from many RHEL servers and went with nft directly. Security Event Auditing Uhm… so like auditd?

cperciva 4y ago
The recent WireGuard debacle left a bad taste about this. As it actually turns out, sometimes there is zero technical review for very important patches and a few blessed developers can (and sometimes do) just throw their stuff directly into trunk. This seems like a misunderstanding of the FreeBSD development model. Yes, immature code landed in HEAD, but it was removed before the next release. In general in FreeBSD there's no expectation that HEAD is always usable. Sometimes it won't even build! It's a place where code can land in the hope that it will be ready by the time the next release rolls around, but "remove code which isn't ready for prime time" isn't an exceptional case. FreeBSD has a very strong history of post-commit code review, largely because every FreeBSD committer gets email when commits go into the tree -- that's a lot of eyeballs. We're moving towards increased pre-commit review thanks now that better tools are available for that, but that's a separate matter. (Yes, Netflix runs FreeBSD HEAD. I think they're nuts.)
- 5e92cb50239222b 4y ago
  Thanks for clarification. Although IIUC the "bad" WireGuard implementation was stopped from being released just barely: https://lwn.net/Articles/850098 https://lwn.net/Articles/850098 It was rushed because a particular vendor wanted to have it as soon as possible. If not for Jason Donenfeld's diligence it looks like we would have out there in the open, full of bugs and all. Doesn't seem like a normal occurrence though, seeing how much noise it made.
  - cperciva 4y ago
    The WG code was introduced unusually late, I agree. Usually experimental stuff like that lands soon after a .0 release so that there's a year to iron out details before the next release. But this is a quantitative difference -- how close to the next release do you push experimental code into the tree -- not a qualitative difference. And fundamentally the system worked! The code was deemed to not be ready and was yanked before the release.
- aaron_m04 4y ago
  > (Yes, Netflix runs FreeBSD HEAD. I think they're nuts.) Sounds like chaos engineering to me ;)
pdimitar 4y ago
> Most services on my machines are heavily locked down and isolated from each other since systemd makes this very easy Do you have a guide handy? A quick Google search only showed generic nonsense. I'll appreciate a recipe-like pointer. Been curious about this for a while.
- 5e92cb50239222b 4y ago
  I haven't seen any decent articles on this (doesn't mean there aren't any) that mention anything besides most basic flags. Have a gander at the comment I linked (https://news.ycombinator.com/item?id=29976096 https://news.ycombinator.com/item?id=29976096). --- Probably the easiest way to start is to create your unit file as usual, copy the list from the comment into it, and then run: $ systemd-analyze security unit-name It prints a huge list of suggestions along with a short description of each one. Look up their names in here: $ man systemd.directives and look at the man linked there. Usually it's one of $ man systemd.exec $ man systemd.resource-control I think the starter list should get you 95% there (I use it for most applications with minor variations in paths and flags like MemoryDenyWriteExecute that breaks JIT compilers). --- If you want to keep your configuration as short as possible, the list with the most bang for your buck would look something like this: User=non-root-username # disable privilege escalation through SUID binaries NoNewPrivileges=yes CapabilityBoundingSet= # removes access to /home ProtectHome=yes # makes most paths read-only ProtectSystem=strict # opens read-write access only to paths your application needs ReadWritePaths=/var/lib/foo /var/lib/bar If your application follows FHS and writes stuff to /var/whatever, prefer: RuntimeDirectory=appname StateDirectory=appname CacheDirectory=appname LogsDirectory=appname ConfigurationDirectory=appname instead of RuntimeDirectory=/var/run/appname StateDirectory=/var/lib/appname CacheDirectory=/var/cache/appname LogsDirectory=/var/log/appname ConfigurationDirectory=/etc/appname
  - pdimitar 4y ago
    Thanks a bunch! I'm absolutely bookmarking your comment and will go back to it.
- prmoustache 4y ago
jsiepkes 4y ago
> ZFS has first-class support on Ubuntu and is compiled into the kernel. And Ubuntu is also the only distribution which has ZFS. Using ZFS on any other distribution (for example RHEL, Rocky Linux, etc.) is a pain. Every update is Russian roulette in which it can break. And everyone except Ubutunu thinks it's a violation of the CDDL. Keep in mind that Oracle is the copyright holder of ZFS. So you (and Ubuntu) are violating Oracle's license terms. Would be realy interesting to see what happens if Oracle decides to sue an Ubuntu user. Would Ubuntu step in to help? > There's also real DTrace on Oracle Linux if you're ready to sell your soul: I can't run Ubuntu (for ZFS) and Oracle Linux (for DTrace) at the same time. Besides, like you said; Why would anyone want to use Oracle's Unbreakable Linux? > Well... linux has containers, and if all you need is isolation... No that's not all I need. I need things like virtual networking between my containers. > I actually think it's worse in this regard because of the link above. Depends. If you put them in a FreeBSD jail they are probably better isolated then only using systemd.
- djbusby 4y ago
  It's not "only Ubuntu". I'm using ZFS (not for boot or root partition) on Gentoo for some time. It's a separate package from kernel, and has to rebuild after a kernel upgrade but it works as expected.
  - presto8 4y ago
    To add another anecdote: I use ZFS on NixOS for all of my partitions, including an encrypted boot partition. It has been rock solid for me.
    - spindle 4y ago
      Not only rock solid but also very easy!
- jen20 4y ago
  > Keep in mind that Oracle is the copyright holder of ZFS. So you (and Ubuntu) are violating Oracle's license terms. If anything IS a problem here it is violating the terms of the GPL, not the CDDL. So no, no one is violating Oracles licensing terms - and if they were, they’d have been sued in 2016 when this shipped. Do you really think ORACLE of all people is just holding back out the goodness of their hearts? Complete FUD.

cperciva 4y ago

The recent WireGuard debacle left a bad taste about this. As it actually turns out, sometimes there is zero technical review for very important patches and a few blessed developers can (and sometimes do) just throw their stuff directly into trunk. This seems like a misunderstanding of the FreeBSD development model. Yes, immature code landed in HEAD, but it was removed before the next release. In general in FreeBSD there's no expectation that HEAD is always usable. Sometimes it won't even build! It's a place where code can land in the hope that it will be ready by the time the next release rolls around, but "remove code which isn't ready for prime time" isn't an exceptional case. FreeBSD has a very strong history of post-commit code review, largely because every FreeBSD committer gets email when commits go into the tree -- that's a lot of eyeballs. We're moving towards increased pre-commit review thanks now that better tools are available for that, but that's a separate matter. (Yes, Netflix runs FreeBSD HEAD. I think they're nuts.)

5e92cb50239222b 4y ago
Thanks for clarification. Although IIUC the "bad" WireGuard implementation was stopped from being released just barely: https://lwn.net/Articles/850098 https://lwn.net/Articles/850098 It was rushed because a particular vendor wanted to have it as soon as possible. If not for Jason Donenfeld's diligence it looks like we would have out there in the open, full of bugs and all. Doesn't seem like a normal occurrence though, seeing how much noise it made.
- cperciva 4y ago
  The WG code was introduced unusually late, I agree. Usually experimental stuff like that lands soon after a .0 release so that there's a year to iron out details before the next release. But this is a quantitative difference -- how close to the next release do you push experimental code into the tree -- not a qualitative difference. And fundamentally the system worked! The code was deemed to not be ready and was yanked before the release.
aaron_m04 4y ago
> (Yes, Netflix runs FreeBSD HEAD. I think they're nuts.) Sounds like chaos engineering to me ;)

pdimitar 4y ago

> Most services on my machines are heavily locked down and isolated from each other since systemd makes this very easy Do you have a guide handy? A quick Google search only showed generic nonsense. I'll appreciate a recipe-like pointer. Been curious about this for a while.

5e92cb50239222b 4y ago
I haven't seen any decent articles on this (doesn't mean there aren't any) that mention anything besides most basic flags. Have a gander at the comment I linked (https://news.ycombinator.com/item?id=29976096 https://news.ycombinator.com/item?id=29976096). --- Probably the easiest way to start is to create your unit file as usual, copy the list from the comment into it, and then run: $ systemd-analyze security unit-name It prints a huge list of suggestions along with a short description of each one. Look up their names in here: $ man systemd.directives and look at the man linked there. Usually it's one of $ man systemd.exec $ man systemd.resource-control I think the starter list should get you 95% there (I use it for most applications with minor variations in paths and flags like MemoryDenyWriteExecute that breaks JIT compilers). --- If you want to keep your configuration as short as possible, the list with the most bang for your buck would look something like this: User=non-root-username # disable privilege escalation through SUID binaries NoNewPrivileges=yes CapabilityBoundingSet= # removes access to /home ProtectHome=yes # makes most paths read-only ProtectSystem=strict # opens read-write access only to paths your application needs ReadWritePaths=/var/lib/foo /var/lib/bar If your application follows FHS and writes stuff to /var/whatever, prefer: RuntimeDirectory=appname StateDirectory=appname CacheDirectory=appname LogsDirectory=appname ConfigurationDirectory=appname instead of RuntimeDirectory=/var/run/appname StateDirectory=/var/lib/appname CacheDirectory=/var/cache/appname LogsDirectory=/var/log/appname ConfigurationDirectory=/etc/appname
- pdimitar 4y ago
  Thanks a bunch! I'm absolutely bookmarking your comment and will go back to it.
prmoustache 4y ago

jsiepkes 4y ago

> ZFS has first-class support on Ubuntu and is compiled into the kernel. And Ubuntu is also the only distribution which has ZFS. Using ZFS on any other distribution (for example RHEL, Rocky Linux, etc.) is a pain. Every update is Russian roulette in which it can break. And everyone except Ubutunu thinks it's a violation of the CDDL. Keep in mind that Oracle is the copyright holder of ZFS. So you (and Ubuntu) are violating Oracle's license terms. Would be realy interesting to see what happens if Oracle decides to sue an Ubuntu user. Would Ubuntu step in to help? > There's also real DTrace on Oracle Linux if you're ready to sell your soul: I can't run Ubuntu (for ZFS) and Oracle Linux (for DTrace) at the same time. Besides, like you said; Why would anyone want to use Oracle's Unbreakable Linux? > Well... linux has containers, and if all you need is isolation... No that's not all I need. I need things like virtual networking between my containers. > I actually think it's worse in this regard because of the link above. Depends. If you put them in a FreeBSD jail they are probably better isolated then only using systemd.

djbusby 4y ago
It's not "only Ubuntu". I'm using ZFS (not for boot or root partition) on Gentoo for some time. It's a separate package from kernel, and has to rebuild after a kernel upgrade but it works as expected.
- presto8 4y ago
  To add another anecdote: I use ZFS on NixOS for all of my partitions, including an encrypted boot partition. It has been rock solid for me.
  - spindle 4y ago
    Not only rock solid but also very easy!
jen20 4y ago
> Keep in mind that Oracle is the copyright holder of ZFS. So you (and Ubuntu) are violating Oracle's license terms. If anything IS a problem here it is violating the terms of the GPL, not the CDDL. So no, no one is violating Oracles licensing terms - and if they were, they’d have been sued in 2016 when this shipped. Do you really think ORACLE of all people is just holding back out the goodness of their hearts? Complete FUD.