3 ms·

Wouldn't it be better to apply stress over more drives to minimize the chances that you lose a second drive during a rebuild?

ErneX 3y ago
I believe this is the article I read when I started: https://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/ https://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs...
- ilyt 3y ago
  That talks more about RAID5/Z1 vs mirroring. You still need 6/Z2 if you want to have reasonable fault tolerance, unless you want to waste ungodly amount of space.
  - ErneX 3y ago
    My reasoning for using mirrored vdevs was more about the ease of expanding the storage than the redundancy.
    - ilyt 3y ago
      We just used RAID6... expanding by more drives is easy while keeping data waste low and the "we want to put bigger drives now" case isn't all that problematic considering that what you save you waste on low efficiency of RAID1 setup. We did had a big case so we just ran 2xRAID6 setup and that one time where it was needed we just replaced to bigger drives one by one, while using the removed ones as spares for other machines. But that's benefit of scale bigger than "a NAS server".
      - ErneX 3y ago
        Yeah mine is a home NAS, not a professional deployment. At first I was a bit disappointed at losing half the storage due to using mirrors but honestly it’s not a big deal considering the prices I managed to find on the drives. And I also wanted to expand at my own pace. The drive I recently used to replace the dead 4TB drive cost me 110€ and it’s a 10TB drive. I got 2 so I’ll also replace the sibling of the dead 4TB one.
- labawi 3y ago
  Note that the listed probabilities of surviving N-disk failures are not correct, though after calculation, though the differences may not be that important. Listed survival probability of N-disk failure in 8-drive/4-vdev mirror. 1: 1; 2: 0.857; 3: 0.667; 4: 0.400; 5: 0; 6: N/A; 7: N/A; 8: N/A Proper survival probability: 1: 1; 2: 0.857; 3: 0.571; 4: 0.229; 5: 0; 6: 0; 7: 0; 8: 0 Comparatively, survival probability for 8-drive RAIDz4 with equal amount of usable space: 1: 1; 2: 1; 3: 1; 4: 1; 5: 0; 6: 0; 7: 0; 8: 0 Personally, I'd use multi-vdev mirror pools only for data that is either backed-up or data I can afford to lose completely.
ilyt 3y ago
To replace drive in RAID5/6 array, you need to read the entirety of every drive To replace drive in RAID1 you need to read the entirety of one drive. Instead of reading one whole drive worth of stress, you're reading N drives worth of stress But it is funny that the people making arguments about "stressing drives" don't even fucking know how RAID works...
- edmundsauto 3y ago
  However, RAID1, you stress the drive that has your only copy of the data. With 5/6 (RAIDz w/ 2 spares as well) you can have multiple copies of the data, so although you are overall increasing stress on your fleet, wouldn't it have a lower probability of data loss? Drive stress (for me) is mostly a concern about data loss.
  - ilyt 3y ago
    With RAID5 you'd have higher probability of data loss. Remember, you just loss the only redundancy you had, brining your redundancy to same level as RAID1 with failed drive. ... except now you have more drives that can fail. With RAID6 yes, you can still fail one more time and still keep your data, which is why it is recommended. Data safety wise I'd go RAID6 -> RAID1/10 (particularly linux implementation can do raid10 on odd number of drives which is nice) -> RAID5

ErneX 3y ago

I believe this is the article I read when I started: https://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/ https://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs...

ilyt 3y ago
That talks more about RAID5/Z1 vs mirroring. You still need 6/Z2 if you want to have reasonable fault tolerance, unless you want to waste ungodly amount of space.
- ErneX 3y ago
  My reasoning for using mirrored vdevs was more about the ease of expanding the storage than the redundancy.
  - ilyt 3y ago
    We just used RAID6... expanding by more drives is easy while keeping data waste low and the "we want to put bigger drives now" case isn't all that problematic considering that what you save you waste on low efficiency of RAID1 setup. We did had a big case so we just ran 2xRAID6 setup and that one time where it was needed we just replaced to bigger drives one by one, while using the removed ones as spares for other machines. But that's benefit of scale bigger than "a NAS server".
    - ErneX 3y ago
      Yeah mine is a home NAS, not a professional deployment. At first I was a bit disappointed at losing half the storage due to using mirrors but honestly it’s not a big deal considering the prices I managed to find on the drives. And I also wanted to expand at my own pace. The drive I recently used to replace the dead 4TB drive cost me 110€ and it’s a 10TB drive. I got 2 so I’ll also replace the sibling of the dead 4TB one.
labawi 3y ago
Note that the listed probabilities of surviving N-disk failures are not correct, though after calculation, though the differences may not be that important. Listed survival probability of N-disk failure in 8-drive/4-vdev mirror. 1: 1; 2: 0.857; 3: 0.667; 4: 0.400; 5: 0; 6: N/A; 7: N/A; 8: N/A Proper survival probability: 1: 1; 2: 0.857; 3: 0.571; 4: 0.229; 5: 0; 6: 0; 7: 0; 8: 0 Comparatively, survival probability for 8-drive RAIDz4 with equal amount of usable space: 1: 1; 2: 1; 3: 1; 4: 1; 5: 0; 6: 0; 7: 0; 8: 0 Personally, I'd use multi-vdev mirror pools only for data that is either backed-up or data I can afford to lose completely.

ilyt 3y ago

To replace drive in RAID5/6 array, you need to read the entirety of every drive To replace drive in RAID1 you need to read the entirety of one drive. Instead of reading one whole drive worth of stress, you're reading N drives worth of stress But it is funny that the people making arguments about "stressing drives" don't even fucking know how RAID works...

edmundsauto 3y ago
However, RAID1, you stress the drive that has your only copy of the data. With 5/6 (RAIDz w/ 2 spares as well) you can have multiple copies of the data, so although you are overall increasing stress on your fleet, wouldn't it have a lower probability of data loss? Drive stress (for me) is mostly a concern about data loss.
- ilyt 3y ago
  With RAID5 you'd have higher probability of data loss. Remember, you just loss the only redundancy you had, brining your redundancy to same level as RAID1 with failed drive. ... except now you have more drives that can fail. With RAID6 yes, you can still fail one more time and still keep your data, which is why it is recommended. Data safety wise I'd go RAID6 -> RAID1/10 (particularly linux implementation can do raid10 on odd number of drives which is nice) -> RAID5