4 ms·

Btrfs is used on millions of production servers (Facebook...) and can essentially solve the same storage problems that ZFS was designed to solve, using similar

Btrfs is used on millions of production servers (Facebook...) and can essentially solve the same storage problems that ZFS was designed to solve, using similar principles (CoW...). Btrfs deservedly got a bad wrap for many years because it had unacceptable bugs, but they seem to have finally quieted. SUSE ships and supports Btrfs for its enterprise Linux, and even Red Hat made it the default fs for Fedora. Also, Oracle can't sue you for using Btrfs, whereas Oracle has refused to have a lawyer send Linus a letter saying Oracle won't sue people for using ZFS on Linux, despite Linus' request to Oracle for such a letter saying its OK.

minimaul 5y ago
btrfs is used by orgs like facebook in a very limited way, and afaik not for big multi-disk storage pools with raid-style devices like ZFS is generally used. btrfs is still very rough for things like raid5/6 (where you want to prioritise available capacity), and unfortunately there are lots of reports of recent corruption (even on single disk FSes) every time we have a thread about it on HN. I'm one of those people with recent experience of corruption with btrfs + compression. ZFS isn't flawless, and the license situation means distribution is more awkward on Linux - except for Ubuntu, who have a different view to other distros on what is allowed with it legally. This means that root on ZFS is a real pain on many distros, and possibly not something you should really do, but ZFS has really solid multi-disk pool support & working + battletested support for things like RAIDZ. But this isn't linux, this is FreeBSD. The license issues with CDDL & GPL don't apply here, so FreeBSD has native ZFS support. It works well (especially in 13, where it's up to date with the main OSS OpenZFS work), and the installer will quite happily assist you with root on ZFS.
- arp242 5y ago
  I'm surprised to learn that btrfs still has corruption issues, even for simple cases. It's been in development for 12 years already; I wonder what the factors behind this are?
  - minimaul 5y ago
    It's disappointing :( I'd love to use btrfs on root on linux, and I try it on my home server and VMs reasonably often (and this is a xeon + ECC etc, and when I use ZFS/xfs/ext4 I don't get FS issues). I keep finding it's often still fragile unfortunately (and snapshotting is still painful compared to ZFS!).
    - KozmoNau7 5y ago
      Conversely, I haven't had a single btrfs issue on my home server, which has gradually expanded from a 2x2TB RAID1. New disks get added to the pool via 'device add' and off we go. That's one of the biggest reasons why I use btrfs, it lets me turn storage into a large generic pool instead of individual partitions and drives, and it's very flexible when combining drives of different sizes. The only issue I did have was a bad ASM1061-based SATA controller, which would corrupt data in weird ways, throw up alignment errors, generic I/O errors, all kinds of issue. That can hardly be blamed on btrfs. Because it was a RAID1 pool and a checksumming filesystem, I could switch the affected drive to a known good SATA controller and correct all of the corrupted data, to bring everything back in line. I found it very robust and undramatic. I strongly assume it would have worked more or less the same on ZFS, but ZFS is less flexible when adding disks (have to add new separate vdevs instead of integrating individual drives), so I stick with btrfs. Don't buy ASM1061-based SATA controllers. At the very least, get something Marvell-based, or an LSI-based SAS controller in IT mode, if you want to get fancy.
      - minimaul 5y ago
        Yeah, I notice some people have very different experiences. It's odd. For me it's a mix of NVMe drives, SATA SSDs on Intel SATA, or SATA SSDs on LSI SAS HBAs (in IT mode). It doesn't seem to be pinned to specific hardware, sadly - and the exact same hardware works fine with zfs/ext4/xfs.
        KozmoNau7 5y ago
        It would be tempting to chalk it up to marginal storage controllers and/or memory or even flaky power supplies. I've seen all three result in very weird and hard to diagnose issues, which presumably a more critical and checksumming filesystem is simply better at sussing out than a more old-fashioned FS like ext4 or XFS. But from the impression of the type of hardware you're running, I highly doubt that is the case. And of course ZFS would be affected, too.
- mekster 5y ago
  I tend not to rely on what facebook does with its software stack. Even by looking at what they release as open source, it's usually "works for me" and not tailored for general public and I think it's in their culture not to care about other use than for themselves. It's hard to think it's a good proof that btrfs is ready for everyone just because facebook is using it.
hnlmorg 5y ago
Comments like yours come up whenever anyone dares discuss ZFS or FreeBSD. Sure, you can use Btrfs, xfs, ext4 or whatever the hell you want on Linux if you want. Nobody is trying to take away your toys. But Linux is already dominant -- you don't need to convince people to use Linux. FreeBSD, however, doesn't get attention it deserves. Hence why threads like these occasionally pop up. And for what it's worth, ZFS development pre-dates Btrfs. You can make all the arguments for Btrfs that you want but ZFS is still the vastly more mature filesystem out of the two. Hence why the few examples people can come up with regards to Btrfs usage are companies with the resources of Facebook (as opposed to ZFS which is used by companies large and small).
trasz 5y ago
Oracle cannot sue anyone for using ZFS, thanks to the patent protection clause in CDDL.
deleted 5y ago
[deleted]

minimaul 5y ago

btrfs is used by orgs like facebook in a very limited way, and afaik not for big multi-disk storage pools with raid-style devices like ZFS is generally used. btrfs is still very rough for things like raid5/6 (where you want to prioritise available capacity), and unfortunately there are lots of reports of recent corruption (even on single disk FSes) every time we have a thread about it on HN. I'm one of those people with recent experience of corruption with btrfs + compression. ZFS isn't flawless, and the license situation means distribution is more awkward on Linux - except for Ubuntu, who have a different view to other distros on what is allowed with it legally. This means that root on ZFS is a real pain on many distros, and possibly not something you should really do, but ZFS has really solid multi-disk pool support & working + battletested support for things like RAIDZ. But this isn't linux, this is FreeBSD. The license issues with CDDL & GPL don't apply here, so FreeBSD has native ZFS support. It works well (especially in 13, where it's up to date with the main OSS OpenZFS work), and the installer will quite happily assist you with root on ZFS.

arp242 5y ago
I'm surprised to learn that btrfs still has corruption issues, even for simple cases. It's been in development for 12 years already; I wonder what the factors behind this are?
- minimaul 5y ago
  It's disappointing :( I'd love to use btrfs on root on linux, and I try it on my home server and VMs reasonably often (and this is a xeon + ECC etc, and when I use ZFS/xfs/ext4 I don't get FS issues). I keep finding it's often still fragile unfortunately (and snapshotting is still painful compared to ZFS!).
  - KozmoNau7 5y ago
    Conversely, I haven't had a single btrfs issue on my home server, which has gradually expanded from a 2x2TB RAID1. New disks get added to the pool via 'device add' and off we go. That's one of the biggest reasons why I use btrfs, it lets me turn storage into a large generic pool instead of individual partitions and drives, and it's very flexible when combining drives of different sizes. The only issue I did have was a bad ASM1061-based SATA controller, which would corrupt data in weird ways, throw up alignment errors, generic I/O errors, all kinds of issue. That can hardly be blamed on btrfs. Because it was a RAID1 pool and a checksumming filesystem, I could switch the affected drive to a known good SATA controller and correct all of the corrupted data, to bring everything back in line. I found it very robust and undramatic. I strongly assume it would have worked more or less the same on ZFS, but ZFS is less flexible when adding disks (have to add new separate vdevs instead of integrating individual drives), so I stick with btrfs. Don't buy ASM1061-based SATA controllers. At the very least, get something Marvell-based, or an LSI-based SAS controller in IT mode, if you want to get fancy.
    - minimaul 5y ago
      Yeah, I notice some people have very different experiences. It's odd. For me it's a mix of NVMe drives, SATA SSDs on Intel SATA, or SATA SSDs on LSI SAS HBAs (in IT mode). It doesn't seem to be pinned to specific hardware, sadly - and the exact same hardware works fine with zfs/ext4/xfs.
      - KozmoNau7 5y ago
        It would be tempting to chalk it up to marginal storage controllers and/or memory or even flaky power supplies. I've seen all three result in very weird and hard to diagnose issues, which presumably a more critical and checksumming filesystem is simply better at sussing out than a more old-fashioned FS like ext4 or XFS. But from the impression of the type of hardware you're running, I highly doubt that is the case. And of course ZFS would be affected, too.
mekster 5y ago
I tend not to rely on what facebook does with its software stack. Even by looking at what they release as open source, it's usually "works for me" and not tailored for general public and I think it's in their culture not to care about other use than for themselves. It's hard to think it's a good proof that btrfs is ready for everyone just because facebook is using it.

hnlmorg 5y ago

Comments like yours come up whenever anyone dares discuss ZFS or FreeBSD. Sure, you can use Btrfs, xfs, ext4 or whatever the hell you want on Linux if you want. Nobody is trying to take away your toys. But Linux is already dominant -- you don't need to convince people to use Linux. FreeBSD, however, doesn't get attention it deserves. Hence why threads like these occasionally pop up. And for what it's worth, ZFS development pre-dates Btrfs. You can make all the arguments for Btrfs that you want but ZFS is still the vastly more mature filesystem out of the two. Hence why the few examples people can come up with regards to Btrfs usage are companies with the resources of Facebook (as opposed to ZFS which is used by companies large and small).

trasz 5y ago

Oracle cannot sue anyone for using ZFS, thanks to the patent protection clause in CDDL.

deleted 5y ago

[deleted]