7 ms·

To be honest, the situation with Linux is barely better. ZFS has license issues with Linux, preventing full integration, and Btrfs is 15 years in the making an

• lousken 2y ago
  as far as stability goes, btrfs is used by meta, synology and many others, so I wouldn't say it's not stable, but some features are lacking
  • fourfour3 2y ago
    Do Synology actually use the multi-device options of btrfs, or are they using linux softraid + lvm underneath? I know Synology Hybrid RAID is a clever use of LVM + MD raid, for example.
    • phs2501 2y ago
      I believe Synology runs btrfs on top of regular mdraid + lvm, possibly with patches to let btrfs checksum failures reach into the underlying layers to find the right data to recover. Related blog post: https://daltondur.st/syno_btrfs_1/ https://daltondur.st/syno_btrfs_1/
      • fourfour3 2y ago
        That was very interesting reading, thanks!
  • azalemeth 2y ago
    My understanding is that single-disk btrfs is good, but raid is decidedly dodgy; https://btrfs.readthedocs.io/en/latest/btrfs-man5.html#raid56-status-and-recommended-practices https://btrfs.readthedocs.io/en/latest/btrfs-man5.html#raid5... states that: > The RAID56 feature provides striping and parity over several devices, same as the traditional RAID5/6. > There are some implementation and design deficiencies that make it unreliable for some corner cases and *the feature should not be used in production, only for evaluation or testing*. > The power failure safety for metadata with RAID56 is not 100%. I have personally been bitten once (about 10 years ago) by btrfs just failing horribly on a single desktop drive. I've used either mdadm + ext4 (for /) or zfs (for large /data mounts) ever since. Zfs is fantastic and I genuinely don't understand why it's not used more widely.
    • lousken 2y ago
      I was assuming OP wants to highlight filesystem use on a workstation/desktop, not for a file server/NAS. I had similar experience decade ago, but these days single drives just work, same with mirroring. For such setups btrfs should be stable. I've never seen a workstation with raid5/6 setup. Secondly, filesystems and volume managers are something else, even if e.g. btrfs and ZFS are essentialy both. For a NAS setup I would still prefer ZFS with truenas scale (or proxmox if virtualization is needed), just because all these scenarios are supported as well. And as far as ZFS goes, encryption is still something I am not sure about especially since I want to use snapshots sending those as a backup to remote machine.
    • hooli_gan 2y ago
      RAID5/6 is not needed with btrfs. One should use RAID1, which supports striping the same data onto multiple drives in a redundant way.
      • radlad 2y ago
        How can you achieve 2-disk fault tolerance using btrfs and RAID 1?
        • 2y ago
  • _joel 2y ago
    I'm similar to some other people here, I guess once they've been bitten by data loss due to btrfs, it's difficult to advocate for it.
    • lousken 2y ago
      I am assuming almost everybody at some point experienced data loss because they pulled out a flash drive too early. Is it safe to assume that we stopped using flash drives because of it?
      • _joel 2y ago
        I'm not sure we have stopped using flash, judging by the pile of USB sticks on my desk :) In relation to the fs analogy if you used a flash drive that you know corrupted your data, you'd throw it away for one you know works.
        • ryao 2y ago
          I once purchased a bunch of flash drives from Google’s online swag store and just unplugging them was often enough to put then in a state where they claimed to be 8MB devices and nothing I wrote to them was ever possible to read back in my limited tests. I stopped using those fast.
  • jeltz 2y ago
    It is possible to corrupt the file system from user space as a normal user with Btrfs. The PostgreSQL devs found that when working on async IO. And as fer as I know that issue has not been fixed. https://www.postgresql.org/message-id/CA%2BhUKGL-sZrfwcdme8jERPbn%2BsGbY13sRCvq8b9Hp%3DhaWpC6fw%40mail.gmail.com https://www.postgresql.org/message-id/CA%2BhUKGL-sZrfwcdme8j...
    • curt15 2y ago
      LMDB users also unearthed a btrfs data corruption bug last year: https://bugzilla.redhat.com/show_bug.cgi?id=2169947 https://bugzilla.redhat.com/show_bug.cgi?id=2169947
• xattt 2y ago
  ZFS on OS X was killed because of Oracle licensing drama. I don’t expect anything better on Windows either.
  • ryao 2y ago
    There is a third party port here: https://openzfsonosx.org/wiki/Main_Page https://openzfsonosx.org/wiki/Main_Page It was actually the NetApp lawsuit that caused problems for Apple’s adoption of ZFS. Apple wanted indemnification from Sun because of the lawsuit, Sun’s CEO did not sign the agreement before Oracle’s acquisition of Sun happened and Oracle had no interest in granting that, so the official Apple port was cancelled. I heard this second hand years later from people who were insiders at Sun.
    • xattt 2y ago
      That’s a shame re: NetApp/ZFS. While third-party ports are great, they lack deep integration that first-party support would have brought (non-kludgy Time Machine which is technically fixed with APFS).
      • deleted 2y ago
        [deleted]
  • BSDobelix 2y ago
    >ZFS on OS X was killed because of Oracle licensing drama. Naa it was Jobs ego not the license: >>Only one person at Steve Jobs' company announces new products: Steve Jobs. https://arstechnica.com/gadgets/2016/06/zfs-the-other-new-apple-file-system-that-almost-was-until-it-wasnt/ https://arstechnica.com/gadgets/2016/06/zfs-the-other-new-ap...
    • bolognafairy 2y ago
      It’s a cute story that plays into the same old assertions about Steve Jobs, but the conclusion is mostly baseless. There are many other, more credible, less conspiratorial, possible explanations.
• mogoh 2y ago
  ZFS might be better then any other FS on Linux (I don't judge that). But you must admit that the situation on Linux is quite better then on Windows. Linux has so many FS in main branch. There is a lot of development. BTRFS had a rocky start, but it got better.
• stephen_g 2y ago
  I’m interested to know what ‘full integration’ does look like, I use ZFS in Proxmox (Debian-based) and it’s really great and super solid, but I haven’t used ZFS in more vanilla Linux distros. Does Proxmox have things that regular Linux is missing out on, or are there shortcomings and things I just don’t realise about Proxmox?
  • whataguy 2y ago
    The difference is that the ZFS kernel module is included by default with Proxmox, whereas with e.g. Debian, you would need to install it manually.
    • pimeys 2y ago
      And you can't follow the latest kernel before the ZFS module supports it.
      • blibble 2y ago
        for Debian that's not exactly a problem
        • oarsinsync 2y ago
          Unless you’re using Debian backports, and they backport a new kernel a week before the zfs backport package update happens. Happened to me more than once. I ended up manually changing the kernel version limitations the second time just to get me back online, but I don’t recall if that ended up hurting me in the long run or not.
      • BSDobelix 2y ago
        Try CachyOS https://cachyos.org/ https://cachyos.org/ , you can even swap from an existing Arch installation: https://wiki-dev.cachyos.org/sk/cachyos_repositories/how_to_add_cachyos_repo/ https://wiki-dev.cachyos.org/sk/cachyos_repositories/how_to_...
      • ryao 2y ago
• BSDobelix 2y ago
  >ZFS has license issues with Linux, preventing full integration No one wants that, openZFS is much healthier without Linux and it's "Foundation/Politics".
  • bhaney 2y ago
    > No one wants that I want that
    • BSDobelix 2y ago
      Then let me tell you that FreeBSD or OmniOS is what you really want ;)
      • bhaney 2y ago
        You're now 0 for 2 at telling me what I want
        • BSDobelix 2y ago
          The customer is not always right, however a good/modern Filesystem really would be something for Linux ;)
          • ruthmarx 2y ago
            > The customer is not always right, An uninvited door-to-door salesman is rarely, if ever right.
            • BSDobelix 2y ago
              HN is more like a Tupperware party. ;)
              • ruthmarx 2y ago
                Well then you ought to go somewhere more appreciative of your pitches ;)
• nabla9 2y ago
  License is not a real issue. It must be just distributed in separate module. No big hurdle.
  • Jnr 2y ago
    From my point of view it is a real usability issue. zfs modules are not in the official repos. You either have to compile it on each machine or use unofficial repos, which is not exactly ideal and can break things if those repos are not up to date. And I guess it also needs some additional steps for secureboot setup on some distros? I really want to try zfs because btrfs has some issues with RAID5 and RAID6 (it is not recommended so I don't use it) but I am not sure I want to risk the overall system stability, I would not want to end up in a situation where my machines don't boot and I have to fix it manually.
    • chillfox 2y ago
      I have been using ZFS on Mint and Alpine Linux for years for all drives (including root) and have never had an issue. It's been fantastic and is super fast. My linux/zfs laptop loads games much faster than an identical machine running Windows. I have never had data corruption issues with ZFS, but I have had both xfs and ext4 destroy entire discs.
    • harshreality 2y ago
      Why are you considering raid5/6? Are you considering building a large storage array? If the data will fit comfortably (50-60% utilization) on one drive, all you need is raid1. Btrfs is fine for raid1 (raid1c3 for extra redundancy); it might have hidden bugs, but no filesystem is immune from those; zfs had a data loss bug (it was rare, but it happened) a year ago. Why use zfs for a boot partition? Unless you're using every disk mounting point and nvme slot for a single large raid array, you can use a cheap 512GB nvme drive or old spare 2.5" ssd for the boot volume. Or two, in btrfs raid1 if you absolutely must... but do you even need redundancy or datasum (which can hurt performance) to protect OS files? Do you really care if static package files get corrupted? Those are easily reinstalled, and modern quality brand SSDs are quite reliable.
• bayindirh 2y ago
  > Most Linux distros still use ext4 by default, which is 19 years old, but ext4 is little more than a series of extensions on top of ext2, which is the same age as NTFS. However, ext4 and XFS are much more simpler and performant than BTRFS & ZFS as root drives on personal systems and small servers. I personally won't use either on a single disk system as root FS, regardless of how fast my storage subsystem is.
  • ryao 2y ago
    ZFS will outscale ext4 in parallel workloads with ease. XFS will often scale better than ext4, but if you use L2ARC and SLOG devices, it is no contest. On top of that, you can use compression for an additional boost. You might also find ZFS outperforms both of them in read workloads on single disks where ARC minimizes cold cache effects. When I began using ZFS for my rootfs, I noticed my desktop environment became more responsive and I attributed that to ARC.
    • bayindirh 2y ago
      No doubt. I want to reiterate my point. Citing myself: > "I personally won't use either on a single disk system as root FS, regardless of how fast my storage subsystem is." (emphasis mine) We are no strangers to filesystems. I personally benchmarked a ZFS7320 extensively, writing a characterization report, plus we have a ZFS7420 for a very long time, complete with separate log SSDs for read and write on every box. However, ZFS is not saturation proof, plus is nowhere near a Lustre cluster performance wise, when scaled. What kills ZFS and BTRFS on desktop systems are write performance, esp. on heavy workloads like system updates. If I need a desktop server (performance-wise), I'd configure it accordingly and use these, but I'd never use BTRFS or ZFS on a single root disk due to their overhead, to reiterate myself thrice.
      • ryao 2y ago
        I am generally happy with the write performance of ZFS. I have not noticed slow system updates on ZFS (although I run Gentoo, so slow is relative here). In what ways is the write performance bad? I am one of the OpenZFS contributors (although I am less active as late). If you bring some deficiency to my attention, there is a chance I might spend the time needed to improve upon it. By the way, ZFS limits the outstanding IO queue depth to try to keep latencies down as a type of QoS, but you can tune it to allow larger IO queue depths, which should improve write performance. If your issue is related to that, it is an area that is known to be able to use improvement in certain situations: https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/ZIO%20Scheduler.html https://openzfs.github.io/openzfs-docs/Performance%20and%20T... https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Module%20Parameters.html#zfs-vdev-async-write-max-active https://openzfs.github.io/openzfs-docs/Performance%20and%20T... https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Module%20Parameters.html#zfs-vdev-sync-write-max-active https://openzfs.github.io/openzfs-docs/Performance%20and%20T...
• cesarb 2y ago
  > Btrfs [...] still doesn't match ZFS in features [...] Isn't the feature in question (array expansion) precisely one which btrfs already had for a long time? Does ZFS have the opposite feature (shrinking the array), which AFAIK btrfs also already had for a long time? (And there's one feature which is important to many, "being in the upstream Linux kernel", that ZFS most likely will never have.)
  • wkat4242 2y ago
    ZFS also had expansion for a long time but it was offline expansion. I don't know if btrfs has also had online for a long time? And shrinking no, that is a big missing feature in ZFS IMO. Understandable considering its heritage (large scale datacenters) but nevertheless an issue for home use. But raidz is rock-solid. Btrfs' raid is not.
    • unsnap_biceps 2y ago
      Raidz wasn't able to be expanded in place before this. You were able to add to a pool that included a raidz vdev, but that raidz vdev was immutable.
      • wkat4242 2y ago
        Oh ok, I've never done this, but I thought it was already there. Maybe this was the original ZFS from Sun? But maybe I just remember it incorrectly, sorry. I've used it on multi-drive arrays but I never had the need for expansion.
        • ryao 2y ago
          You could add top level raidz vdevs or replace the members of a raid-z vdev with larger disks to increase storage space back then. You still have those options now.
  • _gea 2y ago
    You can online expand a ZFS pool for years by replacing disks with larger ones or by adding vdevs. On current OpenZFS you can even expand a Raid-Z by adding disks to it. To shrink a pool, you can remove vdevs. On OpenZFS this is limited to mirror vdevs, native ZFS from Oracle can remove Raid-Z. Kernel integration is not so important as long as use distributions that include ZFS support at distribution level mainly Proxmox and TrueNAS or Ubuntu.
• honestSysAdmin 2y ago
  https://openzfs.github.io/openzfs-docs/Getting%20Started/index.html ZFS runs on all major Linux distros, the source is compiled locally and there is no meaningful license problem. In datacenter and "enterprise" environments we compile ZFS "statically" with other kernel modules all the time. For over six years now, there is an "experimental" option presented by the graphical Ubuntu installer to install the root filesystem on ZFS. Almost everyone I personally know (just my anecdote) chooses this "experimental" option. There has been an occasion here and there of ZFS snapshots taking up too much space, but other than this there have not been any problems. I statically compile ZFS into a kernel that intentionally does not support loading modules on some of my personal laptops. My experience has been great, others' mileage may (certainly will) vary.
• LtdJorge 2y ago
  XFS is 22 and still the best in-tree FS there is :)
• nijave 2y ago
  You've been able to add and remove devices at will for a long time with btrfs (only recently supported in zfs with lots of caveats) Btrfs also supports async/offline dedupe You can also layer it on top of mdadm. Iirc zfs strongly discourages using anything but direct attached physical disks.