6 ms·

I don't think the parts about ZFS being "better integrated" on FreeBSD are really true any more. For one thing, ZFS-on-Linux is now the canonical, upstream Ope

by mappu 7y ago

I don't think the parts about ZFS being "better integrated" on FreeBSD are really true any more. For one thing, ZFS-on-Linux is now the canonical, upstream OpenZFS implementation - https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSOnLinuxNowOpenZFS?showcomments#comments https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSOnLinuxN... For another, Ars Technica recently reviewed FreeBSD 12.1-RELEASE and found poor ZFS integration in the installer and in mount(8): > Eventually, I realized that FreeBSD's mount command didn't really understand my ZFS root filesystem. -- https://arstechnica.com/gadgets/2020/04/not-actually-linux-distro-review-freebsd-12-1-release/ https://arstechnica.com/gadgets/2020/04/not-actually-linux-d... Compare that with the relatively streamlined ZFS-on-root install support built into Ubuntu: https://arstechnica.com/information-technology/2019/10/a-detailed-look-at-ubuntus-new-experimental-zfs-installer/ https://arstechnica.com/information-technology/2019/10/a-det... (I'm sticking with ext4 until kvdo, bcachefs, or maybe tux3 is mainlined.)

viraptor 7y ago
> (I'm sticking with ext4 until kvdo, bcachefs, or maybe tux3 is mainlined.) I haven't heard of kvdo before - thanks for bringing it up! I'll add stratis-storage to this mix too. (Although I'm contributing to the bcachefs development and really want it to succeed)
rashkov 7y ago
why not use zfs, since it does the things that kvdo, bcachefs, and tux3?
- dragonsh 7y ago
  Only one reason of Oracle law suite sword hanging over neck. Even if canonical gives guarantee they will fight with Oracle, if a small company is dragged by oracle in a court room it's hard to fathom the expenses. So yes there is a need for zfs alternative. So far stayed away from it and using btrfs which works well, except for some RAID limitations.
  - p_l 7y ago
    Fortunately the license Sun applied means any lawsuit would need to attack something much more fundamental, like the very idea of interoperability, as they don't have option to "claw back" any portion of the OpenZFS codebase.
    - takeda 7y ago
      Given that they are currently fighting that an API should be copyrightable, doesn't make me confident that they won't do this.
      - p_l 7y ago
        If they do, GPL is fucked anyway. And not just GPL.
        dragonsh 7y ago
        ZFS is not released under GPL, if it is then it won't be a problem. So there is a need for an alternative and BTRFS works well for me, no need of license encumbered ZFS, not sure of others.
- mappu 7y ago
  My personal reason is that zfs.ko entirely bypasses the kernel page cache, preferring its own internal cache (the ARC). Reimplementing kernel subsystems is a recipe for interoperability nightmare (c.f. the NVIDIA binary driver). For instance, this causes unnecessary memory pressure when you have a mix of ZFS and non-ZFS filesystems (e.g. SMB mounts). Additionally I curmudgeonly have yet to come to terms with the blatant layering violation of ZFS being a combined block-device-manager and filesystem. But the benefits (file-level RAID behaviour) of that are indeed clear.
  - josteink 7y ago
    > I curmudgeonly have yet to come to terms with the blatant layering violation of ZFS being a combined block-device-manager and filesystem It has both advantages and disadvantages. It’s a trade-off. ZFS decided it was worth it, and so did btrfs.
  - takeda 7y ago
    Sure they committed these crimes, but those are the reasons why ZFS is better than the alternatives.
asveikau 7y ago
> Eventually, I realized that FreeBSD's mount command didn't really understand my ZFS root filesystem. I couldn't find this direct quote in your link. But I'm using FreeBSD with ZFS on a few machines and I don't understand this comment. You can absolutely boot with ZFS on root. The installer lets you do it out of the box. Including with geli for encryption. And you can use mount(8) to mount a ZFS partition, I have done that all the time when booting from recovery media and mounting my root pool. This gets a little confusing at first because most uses of mount(8) require a device name, and mounting ZFS takes a pool name, so you may have to zpool import first, but once you get over that it's fine.
- mappu 7y ago
  The quote appears in the article, on page 3 after some lazy-loading (do you have Javascript enabled?)
  - asveikau 7y ago
    Ah ok, this is about trying to remount read-write. Yes, that requires zfs set readonly=off on FreeBSD. [vs. -o rw with mount(8)].
    - takeda 7y ago
      Actually zfs mount -u / works just fine.
      - asveikau 7y ago
        But I think the complaint is that it doesn't happen through mount(8).
kaylynb 7y ago
The Linux kernel devs hate ZFS and actively sabotage it even backporting changes specifically to break ZFS to LTS branches. It's fine if you use something like Ubuntu where it's officially shipped with the distro. If not, be prepared to apply kernel patches whenever the kernel team takes a snipe at ZFS. I'm not trying to be political here, just my day-to-day experience working with several Linux machines running large zpools. It's still worth it compared to the alternatives but if I could use FreeBSD I would.
- _emacsomancer_ 7y ago
  The Void team does a good job with managing ZFS as well (it's in the official repos). The unofficial ZFS on Arch is generally ok as well (though Arch's management of kernels is weird and a bit tricky).
- goatinaboat 7y ago
  Do you have an example of a commit where they did this? Just curious as to how (no need to explain why, I get it)
  - magicalhippo 7y ago
    See https://github.com/openzfs/zfs/issues/8259 https://github.com/openzfs/zfs/issues/8259 for details. While they didn't remove it to sabotage ZFS, they don't seem to care about it ruining ZFS performance on Linux. Of course they have no obligation to make ZFS on Linux users happy, still a bit of a bummer tho.
- leni536 7y ago
  It looks like Linux kernel devs just don't care about ZFS, like they don't care about many out-of-tree kernel patches/modules.
- AsyncAwait 7y ago
  There's absolutely no evidence for this. On the other hand, there's some evidence that Sun made ZFS licensing explicitly non-compatible with the GPL, so as to not "sabotage" Solaris. Linux has other efforts going on in this area, including Btrfs and Bcachefs - there's also ZoL, of course.
petre 7y ago
> I don't think the parts about ZFS being "better integrated" on FreeBSD are really true any more. ZoL is not in the mainline Linux kernel, therefore it isn't even "integrated". Linux 5.x introduced some backwards incompatible changes which broke ZoL also triggering a response from Linus. https://www.phoronix.com/scan.php?page=news_item&px=Linus-Says-No-To-ZFS-Linux https://www.phoronix.com/scan.php?page=news_item&px=Linus-Sa... For these reasons I'm also going to stick with ext4 on Linux. In the meantime we run appliances and servers which run ZFS on FreeBSD without any issues. SmartOS is another choice to run ZFS but I don't have any experience using it. Maybe I'll try it as an alternative to Proxmox.

by mappu 7y ago

viraptor 7y ago

> (I'm sticking with ext4 until kvdo, bcachefs, or maybe tux3 is mainlined.) I haven't heard of kvdo before - thanks for bringing it up! I'll add stratis-storage to this mix too. (Although I'm contributing to the bcachefs development and really want it to succeed)

rashkov 7y ago

why not use zfs, since it does the things that kvdo, bcachefs, and tux3?

dragonsh 7y ago
Only one reason of Oracle law suite sword hanging over neck. Even if canonical gives guarantee they will fight with Oracle, if a small company is dragged by oracle in a court room it's hard to fathom the expenses. So yes there is a need for zfs alternative. So far stayed away from it and using btrfs which works well, except for some RAID limitations.
- p_l 7y ago
  Fortunately the license Sun applied means any lawsuit would need to attack something much more fundamental, like the very idea of interoperability, as they don't have option to "claw back" any portion of the OpenZFS codebase.
  - takeda 7y ago
    Given that they are currently fighting that an API should be copyrightable, doesn't make me confident that they won't do this.
    - p_l 7y ago
      If they do, GPL is fucked anyway. And not just GPL.
      - dragonsh 7y ago
        ZFS is not released under GPL, if it is then it won't be a problem. So there is a need for an alternative and BTRFS works well for me, no need of license encumbered ZFS, not sure of others.
mappu 7y ago
My personal reason is that zfs.ko entirely bypasses the kernel page cache, preferring its own internal cache (the ARC). Reimplementing kernel subsystems is a recipe for interoperability nightmare (c.f. the NVIDIA binary driver). For instance, this causes unnecessary memory pressure when you have a mix of ZFS and non-ZFS filesystems (e.g. SMB mounts). Additionally I curmudgeonly have yet to come to terms with the blatant layering violation of ZFS being a combined block-device-manager and filesystem. But the benefits (file-level RAID behaviour) of that are indeed clear.
- josteink 7y ago
  > I curmudgeonly have yet to come to terms with the blatant layering violation of ZFS being a combined block-device-manager and filesystem It has both advantages and disadvantages. It’s a trade-off. ZFS decided it was worth it, and so did btrfs.
- takeda 7y ago
  Sure they committed these crimes, but those are the reasons why ZFS is better than the alternatives.

asveikau 7y ago

> Eventually, I realized that FreeBSD's mount command didn't really understand my ZFS root filesystem. I couldn't find this direct quote in your link. But I'm using FreeBSD with ZFS on a few machines and I don't understand this comment. You can absolutely boot with ZFS on root. The installer lets you do it out of the box. Including with geli for encryption. And you can use mount(8) to mount a ZFS partition, I have done that all the time when booting from recovery media and mounting my root pool. This gets a little confusing at first because most uses of mount(8) require a device name, and mounting ZFS takes a pool name, so you may have to zpool import first, but once you get over that it's fine.

mappu 7y ago
The quote appears in the article, on page 3 after some lazy-loading (do you have Javascript enabled?)
- asveikau 7y ago
  Ah ok, this is about trying to remount read-write. Yes, that requires zfs set readonly=off on FreeBSD. [vs. -o rw with mount(8)].
  - takeda 7y ago
    Actually zfs mount -u / works just fine.
    - asveikau 7y ago
      But I think the complaint is that it doesn't happen through mount(8).

kaylynb 7y ago

The Linux kernel devs hate ZFS and actively sabotage it even backporting changes specifically to break ZFS to LTS branches. It's fine if you use something like Ubuntu where it's officially shipped with the distro. If not, be prepared to apply kernel patches whenever the kernel team takes a snipe at ZFS. I'm not trying to be political here, just my day-to-day experience working with several Linux machines running large zpools. It's still worth it compared to the alternatives but if I could use FreeBSD I would.

_emacsomancer_ 7y ago
The Void team does a good job with managing ZFS as well (it's in the official repos). The unofficial ZFS on Arch is generally ok as well (though Arch's management of kernels is weird and a bit tricky).
goatinaboat 7y ago
Do you have an example of a commit where they did this? Just curious as to how (no need to explain why, I get it)
- magicalhippo 7y ago
  See https://github.com/openzfs/zfs/issues/8259 https://github.com/openzfs/zfs/issues/8259 for details. While they didn't remove it to sabotage ZFS, they don't seem to care about it ruining ZFS performance on Linux. Of course they have no obligation to make ZFS on Linux users happy, still a bit of a bummer tho.
leni536 7y ago
It looks like Linux kernel devs just don't care about ZFS, like they don't care about many out-of-tree kernel patches/modules.
AsyncAwait 7y ago
There's absolutely no evidence for this. On the other hand, there's some evidence that Sun made ZFS licensing explicitly non-compatible with the GPL, so as to not "sabotage" Solaris. Linux has other efforts going on in this area, including Btrfs and Bcachefs - there's also ZoL, of course.

petre 7y ago

> I don't think the parts about ZFS being "better integrated" on FreeBSD are really true any more. ZoL is not in the mainline Linux kernel, therefore it isn't even "integrated". Linux 5.x introduced some backwards incompatible changes which broke ZoL also triggering a response from Linus. https://www.phoronix.com/scan.php?page=news_item&px=Linus-Says-No-To-ZFS-Linux https://www.phoronix.com/scan.php?page=news_item&px=Linus-Sa... For these reasons I'm also going to stick with ext4 on Linux. In the meantime we run appliances and servers which run ZFS on FreeBSD without any issues. SmartOS is another choice to run ZFS but I don't have any experience using it. Maybe I'll try it as an alternative to Proxmox.