14 ms·

One that seems to catch out many people is that you need to explicitly add the "degraded" option to allow mounting raid volumes which contain a broken drive. Th

One that seems to catch out many people is that you need to explicitly add the "degraded" option to allow mounting raid volumes which contain a broken drive. This is opposite to almost all other filesystems. I've seen people confused and thinking they lost the whole volume, even though they just need to replace the bad drive and rebuild as usual.

ndsipa_pomu 3y ago
Personally, I agree with that. It's good practise for systems to fail quickly if they find themselves in an uncertain state and with RAID, it's important for the operator to know if they've suddenly lost redundancy so that they can resolve the issue.
- viraptor 3y ago
  It really depends on what's your situation. At home I probably want the disks to stay idle until replacement. In a bigger production system I want to keep the availability and ping the on-call person to replace the drive in the background.
  - ndsipa_pomu 3y ago
    Exactly - the default should be to get someone to pay attention to it when something breaks, and if you're planning on high availability, then you can choose that, assuming you keep an eye on the health of the RAID.
- keep_reading 3y ago
  If you can't even boot because of a mystery error how are you supposed to resolve the issue? It should instead be giving the user error messages written to their terminal, in logs, etc instead of breaking the entire system until the user finds the manual
  - wtallis 3y ago
    If your NAS has a hot spare drive installed then it should probably include the degraded mount option by default and automatically add the hot spare drive to the filesystem in the event of a failure. Alternatively, if there's enough free space on the surviving drives to rebalance the array and restore redundancy without replacing the failed drive, that operation could be kicked off automatically. Or the filesystem can be (hopefully temporarily) set to not store new data redundantly, if that is an acceptable risk for the user. But the filesystem cannot know which method the user would prefer; automatically rebuilding the array involves policy decisions that are outside the scope of the filesystem and requires userspace tooling. If the system doesn't have spare capacity ready, the only sane response is to not boot/mount normally. "giving the user error messages written to their terminal, in logs, etc" isn't a real solution for something like a NAS with no terminal connected and nobody looking at the logs as long as they can still establish a SMB connection; it's too likely to be a silent failure in practice. Mounting the filesystem degraded but read-only makes sense if it's necessary to boot the system so that the user (or their pre-configured userspace tooling) can decide how to deal with the problem, but a lot of Linux distros aren't happy with the root filesystem being read-only. In summary: there's no single right answer to the problem of a failed drive, and btrfs defaults to what is the safest behavior based on the information available to the filesystem itself. Userspace tooling with more information can make other, less universal choices. A distro that tries to simply adopt btrfs as a drop-in replacement for ext4 probably doesn't have all the tooling necessary to make good use of the unique features of btrfs.
    - keep_reading 3y ago
      > If the system doesn't have spare capacity ready, the only sane response is to not boot/mount normally. It doesn't need the spare to "boot normally" and the system can turn on a scary LED, ring bells, call you, text you, hit you up on WhatsApp, DM you on Instagram, or whatever method you want your NAS to use to notify you there's a degradation. (You're monitoring it right??) This explanation of "it's dangerous to boot off a degraded array" is lunacy. I will not take this terrible advice from armchair experts when I've been doing this for over 25 years

ndsipa_pomu 3y ago

Personally, I agree with that. It's good practise for systems to fail quickly if they find themselves in an uncertain state and with RAID, it's important for the operator to know if they've suddenly lost redundancy so that they can resolve the issue.

viraptor 3y ago
It really depends on what's your situation. At home I probably want the disks to stay idle until replacement. In a bigger production system I want to keep the availability and ping the on-call person to replace the drive in the background.
- ndsipa_pomu 3y ago
  Exactly - the default should be to get someone to pay attention to it when something breaks, and if you're planning on high availability, then you can choose that, assuming you keep an eye on the health of the RAID.
keep_reading 3y ago
If you can't even boot because of a mystery error how are you supposed to resolve the issue? It should instead be giving the user error messages written to their terminal, in logs, etc instead of breaking the entire system until the user finds the manual
- wtallis 3y ago
  If your NAS has a hot spare drive installed then it should probably include the degraded mount option by default and automatically add the hot spare drive to the filesystem in the event of a failure. Alternatively, if there's enough free space on the surviving drives to rebalance the array and restore redundancy without replacing the failed drive, that operation could be kicked off automatically. Or the filesystem can be (hopefully temporarily) set to not store new data redundantly, if that is an acceptable risk for the user. But the filesystem cannot know which method the user would prefer; automatically rebuilding the array involves policy decisions that are outside the scope of the filesystem and requires userspace tooling. If the system doesn't have spare capacity ready, the only sane response is to not boot/mount normally. "giving the user error messages written to their terminal, in logs, etc" isn't a real solution for something like a NAS with no terminal connected and nobody looking at the logs as long as they can still establish a SMB connection; it's too likely to be a silent failure in practice. Mounting the filesystem degraded but read-only makes sense if it's necessary to boot the system so that the user (or their pre-configured userspace tooling) can decide how to deal with the problem, but a lot of Linux distros aren't happy with the root filesystem being read-only. In summary: there's no single right answer to the problem of a failed drive, and btrfs defaults to what is the safest behavior based on the information available to the filesystem itself. Userspace tooling with more information can make other, less universal choices. A distro that tries to simply adopt btrfs as a drop-in replacement for ext4 probably doesn't have all the tooling necessary to make good use of the unique features of btrfs.
  - keep_reading 3y ago
    > If the system doesn't have spare capacity ready, the only sane response is to not boot/mount normally. It doesn't need the spare to "boot normally" and the system can turn on a scary LED, ring bells, call you, text you, hit you up on WhatsApp, DM you on Instagram, or whatever method you want your NAS to use to notify you there's a degradation. (You're monitoring it right??) This explanation of "it's dangerous to boot off a degraded array" is lunacy. I will not take this terrible advice from armchair experts when I've been doing this for over 25 years