4 ms·

or install BTRFS and be able to detect bitrot as well, but also be able to grow your storage by adding more disks. (even different sizes)

Geezus_42 6y ago
Just avoid RAID 5/6...
Mister_Snuggles 6y ago
I've been burned by BTRFS too many times to trust it. At work, our servers will get into a state where they just hang for a span from minutes to hours while BTRFS does "something". I'm not one of the admins though, so I don't know the exact details. I just know that this is a vendor-supported configuration and the vendor has been unable to tell us why this happens or offer any solution that makes it not happen. Our answer to this issue has been to rebuild servers with ext4 when things get bad. This has happened on multiple servers hosting different applications - the only commonality seems to be that write-heavy loads get it into this state. Servers that just have their OS on BTRFS but do all of their work on NFS volumes are fine. At home, I once rebooted my OpenSuse Tumbleweed laptop and ended up with a BTRFS filesystem that couldn't be mounted RW. Fortunately I was able to mount it RO after booting off installation media and copy my data off, but I couldn't get the filesystem back into a state where it could be mounted RW. I ended up reinstalling. I never did figure out the root cause, but I suspect that some BTRFS-related process was running when I rebooted. On the flip side, ZFS has never let me down in this way, but to be fair I've never subjected it to the same use-cases. Unfortunately, the inability to resize/reshape the filesystem is an issue for me. I believe that it's being worked on, but I don't think that work is production-ready yet.
tw04 6y ago
RAID-Z expansion is being worked on: https://github.com/openzfs/zfs/pull/8853 https://github.com/openzfs/zfs/pull/8853 Last I checked BTRFS RAID5/6 was a dumpster fire and unusable in production. Have they actually open sourced the ability to fix bitrot detection with mdraid? If not, it's kind of irrelevant. So... once again down votes without response - BTRFS raid still isn't recommended and the file healing isn't compatible with MDRAID I assume and you just don't like the fact I pointed it out? The "I'm downvoting because you pointed out a flaw in my logic" @HN is disappointing.
- accelbred 6y ago
  If you're using RAID-Z on zfs, your comparison isn't fair. Rather than use RAID56 with btrfs, the equivalent would be to get 1 or 2 disk redundancy with raid1 or raid1c3.
  - trasz 6y ago
    Not really - RAIDZ is basically RAID5 without the write hole problem. ZFS equivalent to RAID1 is called 'mirror', and is... well, a mirror.
    - accelbred 6y ago
      Yes, and btrfs's raid1 isn't a mirror. Btrfs's raid1c3 and raid1c4 are its alternative to raid5/6 without the write hole.
      - deleted 6y ago
        [deleted]
      - trasz 6y ago
        No, it's still a mirror, just spread over more than two devices. Size overhead is still that of a mirror, ie quite a bit higher than that of raidz.
  - tw04 6y ago
    RAID-Z is the equivalent of RAID-5. RAID-Z2 is the equivalent of RAID-6. RAID-Z3 would be the equivalent of RAID-7 (or whatever the standard is named for 3-disk parity). This is strictly speaking to how it deals with data and parity, the implementations are obviously different. RAID-1 would be a mirror in ZFS parlance.
    - accelbred 6y ago
      BTRFS raid1 isn't mirroring drives though, it means there are two copies of each extent across the whole set of 2+ drives. and BTRFS and raid1c3 and raid1c4 are 3 and 4 copies.
Geezus_42 6y ago
Just avoid RAID 5/6

Geezus_42 6y ago

Just avoid RAID 5/6...

Mister_Snuggles 6y ago

I've been burned by BTRFS too many times to trust it. At work, our servers will get into a state where they just hang for a span from minutes to hours while BTRFS does "something". I'm not one of the admins though, so I don't know the exact details. I just know that this is a vendor-supported configuration and the vendor has been unable to tell us why this happens or offer any solution that makes it not happen. Our answer to this issue has been to rebuild servers with ext4 when things get bad. This has happened on multiple servers hosting different applications - the only commonality seems to be that write-heavy loads get it into this state. Servers that just have their OS on BTRFS but do all of their work on NFS volumes are fine. At home, I once rebooted my OpenSuse Tumbleweed laptop and ended up with a BTRFS filesystem that couldn't be mounted RW. Fortunately I was able to mount it RO after booting off installation media and copy my data off, but I couldn't get the filesystem back into a state where it could be mounted RW. I ended up reinstalling. I never did figure out the root cause, but I suspect that some BTRFS-related process was running when I rebooted. On the flip side, ZFS has never let me down in this way, but to be fair I've never subjected it to the same use-cases. Unfortunately, the inability to resize/reshape the filesystem is an issue for me. I believe that it's being worked on, but I don't think that work is production-ready yet.

tw04 6y ago

RAID-Z expansion is being worked on: https://github.com/openzfs/zfs/pull/8853 https://github.com/openzfs/zfs/pull/8853 Last I checked BTRFS RAID5/6 was a dumpster fire and unusable in production. Have they actually open sourced the ability to fix bitrot detection with mdraid? If not, it's kind of irrelevant. So... once again down votes without response - BTRFS raid still isn't recommended and the file healing isn't compatible with MDRAID I assume and you just don't like the fact I pointed it out? The "I'm downvoting because you pointed out a flaw in my logic" @HN is disappointing.

accelbred 6y ago
If you're using RAID-Z on zfs, your comparison isn't fair. Rather than use RAID56 with btrfs, the equivalent would be to get 1 or 2 disk redundancy with raid1 or raid1c3.
- trasz 6y ago
  Not really - RAIDZ is basically RAID5 without the write hole problem. ZFS equivalent to RAID1 is called 'mirror', and is... well, a mirror.
  - accelbred 6y ago
    Yes, and btrfs's raid1 isn't a mirror. Btrfs's raid1c3 and raid1c4 are its alternative to raid5/6 without the write hole.
    - deleted 6y ago
      [deleted]
    - trasz 6y ago
      No, it's still a mirror, just spread over more than two devices. Size overhead is still that of a mirror, ie quite a bit higher than that of raidz.
- tw04 6y ago
  RAID-Z is the equivalent of RAID-5. RAID-Z2 is the equivalent of RAID-6. RAID-Z3 would be the equivalent of RAID-7 (or whatever the standard is named for 3-disk parity). This is strictly speaking to how it deals with data and parity, the implementations are obviously different. RAID-1 would be a mirror in ZFS parlance.
  - accelbred 6y ago
    BTRFS raid1 isn't mirroring drives though, it means there are two copies of each extent across the whole set of 2+ drives. and BTRFS and raid1c3 and raid1c4 are 3 and 4 copies.

Geezus_42 6y ago

Just avoid RAID 5/6