5 ms·

Raid 1 usually means a mirrored disk - how does this work with 4 drives? Staying away from unstable features: Is RAID 5 & 6 really considered unstable still?

Shish2k 6y ago
> Raid 1 usually means a mirrored disk - how does this work with 4 drives? “RAID1” in btrfs means “two copies of each file are stored on two disks” - so it’s closer to RAID10 in mdadm terms. It’s pretty nice as you can do things like having 3 x 1TB drives + 1 x 3TB drive and it Just Works to give 3TB of usable space. > Staying away from unstable features: Is RAID 5 & 6 really considered unstable still? Yup :( Also last time I checked (I’d love to be corrected about this) even RAID1 still isn’t ideal (in particular, when a drive fails in a RAID1 array, you only get one chance to mount it r/w and rebalance your data - after that, the array is read-only, and you need to recover by creating a new array on new disks and copy all the data across)
- 8fingerlouie 6y ago
  > Also last time I checked (I’d love to be corrected about this) even RAID1 still isn’t ideal (in particular, when a drive fails in a RAID1 array, you only get one chance to mount it r/w and rebalance your data - after that, the array is read-only, and you need to recover by creating a new array on new disks and copy all the data across) I think you're referring to this (https://www.spinics.net/lists/linux-btrfs/msg63370.html https://www.spinics.net/lists/linux-btrfs/msg63370.html) bug, which was fixed in kernel >= 4.20 and >= 5.0 (according to https://lore.kernel.org/linux-btrfs/CAJCQCtTRseEwoN4cbsAaE_YZz5hUYF1oCPB-aRvz7q2mYWJfMw@mail.gmail.com/t/#m38b32c8f3c53cb55bdd4d012006407647ac6cda1 https://lore.kernel.org/linux-btrfs/CAJCQCtTRseEwoN4cbsAaE_Y...) That being said, the bug only causes the filesystem to become permanently degraded if the number of devices fall below the required number of devices needed for the raid type, and IIRC was caused by btrfs being unable to write metadata correctly. If you had a 3 drive raid1 pool, and one of the drives failed, it would happily continue working in degraded state.
- pas 6y ago
  In case of brtfs with multiple devices it's possible to remove a failed/missing one. 1. unmount the FS 2. make sure the dead/missing/failing device is disconnected 3. mount using -o degraded option. 4. run btrfs device remove missing <mount point> It depends on having at least as many good drives as required for mounting the FS (which is 2 for raid1). https://www.reddit.com/r/btrfs/comments/7os094/removing_a_device/dsbte3d/ https://www.reddit.com/r/btrfs/comments/7os094/removing_a_de...
archi42 6y ago
IIRC btrfs operates on chunks of 1gb instead on the drives themselves. So each chunk is present on two drives for raid1. Data point: I have 10 disks between 1 and 4 TB in a single btrfs raid1 (failure rates don't matter for that private build-from-scrap backup server with a few hours uptime/week). Don't know if raid5/6 are still considered unstable, but would be very interested as well. //edit: shish2k was a bit faster and says nope to raid5/6
MayeulC 6y ago
Not sure how up-to-date that wiki page is: https://btrfs.wiki.kernel.org/index.php/RAID56 https://btrfs.wiki.kernel.org/index.php/RAID56 But TBH, I am fine with some down time in case of a drive failure on my personal server. I try to maintain up-to-date backups anyway, and that's a low-probability event, especially without many drives.

Shish2k 6y ago

> Raid 1 usually means a mirrored disk - how does this work with 4 drives? “RAID1” in btrfs means “two copies of each file are stored on two disks” - so it’s closer to RAID10 in mdadm terms. It’s pretty nice as you can do things like having 3 x 1TB drives + 1 x 3TB drive and it Just Works to give 3TB of usable space. > Staying away from unstable features: Is RAID 5 & 6 really considered unstable still? Yup :( Also last time I checked (I’d love to be corrected about this) even RAID1 still isn’t ideal (in particular, when a drive fails in a RAID1 array, you only get one chance to mount it r/w and rebalance your data - after that, the array is read-only, and you need to recover by creating a new array on new disks and copy all the data across)

8fingerlouie 6y ago
> Also last time I checked (I’d love to be corrected about this) even RAID1 still isn’t ideal (in particular, when a drive fails in a RAID1 array, you only get one chance to mount it r/w and rebalance your data - after that, the array is read-only, and you need to recover by creating a new array on new disks and copy all the data across) I think you're referring to this (https://www.spinics.net/lists/linux-btrfs/msg63370.html https://www.spinics.net/lists/linux-btrfs/msg63370.html) bug, which was fixed in kernel >= 4.20 and >= 5.0 (according to https://lore.kernel.org/linux-btrfs/CAJCQCtTRseEwoN4cbsAaE_YZz5hUYF1oCPB-aRvz7q2mYWJfMw@mail.gmail.com/t/#m38b32c8f3c53cb55bdd4d012006407647ac6cda1 https://lore.kernel.org/linux-btrfs/CAJCQCtTRseEwoN4cbsAaE_Y...) That being said, the bug only causes the filesystem to become permanently degraded if the number of devices fall below the required number of devices needed for the raid type, and IIRC was caused by btrfs being unable to write metadata correctly. If you had a 3 drive raid1 pool, and one of the drives failed, it would happily continue working in degraded state.
pas 6y ago
In case of brtfs with multiple devices it's possible to remove a failed/missing one. 1. unmount the FS 2. make sure the dead/missing/failing device is disconnected 3. mount using -o degraded option. 4. run btrfs device remove missing <mount point> It depends on having at least as many good drives as required for mounting the FS (which is 2 for raid1). https://www.reddit.com/r/btrfs/comments/7os094/removing_a_device/dsbte3d/ https://www.reddit.com/r/btrfs/comments/7os094/removing_a_de...

archi42 6y ago

IIRC btrfs operates on chunks of 1gb instead on the drives themselves. So each chunk is present on two drives for raid1. Data point: I have 10 disks between 1 and 4 TB in a single btrfs raid1 (failure rates don't matter for that private build-from-scrap backup server with a few hours uptime/week). Don't know if raid5/6 are still considered unstable, but would be very interested as well. //edit: shish2k was a bit faster and says nope to raid5/6

MayeulC 6y ago

Not sure how up-to-date that wiki page is: https://btrfs.wiki.kernel.org/index.php/RAID56 https://btrfs.wiki.kernel.org/index.php/RAID56 But TBH, I am fine with some down time in case of a drive failure on my personal server. I try to maintain up-to-date backups anyway, and that's a low-probability event, especially without many drives.