4 ms·

> It is not safe to have more than 8 drives in an array. What are you basing this claim on? Did you mean to qualify that statement to only apply to certain RAI

> It is not safe to have more than 8 drives in an array. What are you basing this claim on? Did you mean to qualify that statement to only apply to certain RAID modes, or are you again just repeating advice specific to ZFS as if it applies to all storage systems?

bluGill 5y ago
All raid. Your odds of a dual fail failure are too high. Note, good raid6 should have x and y stripes with no more than 8 drives in any stripe. I have no idea what they do, so maybe they can restripe somehow on one drive. Hiding the implementation detail is good if they do this right. But I don't know how to tell if any do it right.
- wtallis 5y ago
  > All raid. Your odds of a dual fail failure are too high. These two sentences are in conflict. There are RAID modes where the failure of two drives is recoverable. You probably know this, since you mention one of them in your next sentence. But maybe you don't realize that RAID 6 isn't the only way for an array to survive the failure of two drives, especially if the failures aren't simultaneous. You should read up on some storage systems beyond traditional hardware RAID and ZFS; there are quite a few interesting ideas out there you should open your mind to. Btrfs would be a great place to start your investigation.
- Dylan16807 5y ago
  If 8 drives is safe with dual parity, then significantly more than 8 drives would be safe with triple parity, right?
  - Dylan16807 5y ago
    I did some calculations, with an estimated 3% annual failure rate per drive. The numbers shift a bit depending on how much you expect drive failures to correlate and how long you take to replace them. So I analyzed two example cases, using pretty good math based on a binomial distribution. The first case is a pessimistic one, where after a drive fails we assume that any other possible failures from the next year will hit before we finish rebuilding. In this case, 8 drives with double parity have a 1/740 chance of array failure each year. If we switch to triple parity, we can have 16 drives and a slight improvement in safety. The second case is a more optimistic one, where after a drive fails we assume that any other possible failures from the next week will hit before we finish rebuilding. In this case, 8 drives with double parity have a 1 / 93 million chance of array failure each week. If we switch to triple parity, we can have 40 drives and a slight improvement in safety. I would have done "what if all the drive failures happened at once" but that gives >10% odds of an 8 drive array failing, which seems unreasonably harsh.

bluGill 5y ago

All raid. Your odds of a dual fail failure are too high. Note, good raid6 should have x and y stripes with no more than 8 drives in any stripe. I have no idea what they do, so maybe they can restripe somehow on one drive. Hiding the implementation detail is good if they do this right. But I don't know how to tell if any do it right.

wtallis 5y ago
> All raid. Your odds of a dual fail failure are too high. These two sentences are in conflict. There are RAID modes where the failure of two drives is recoverable. You probably know this, since you mention one of them in your next sentence. But maybe you don't realize that RAID 6 isn't the only way for an array to survive the failure of two drives, especially if the failures aren't simultaneous. You should read up on some storage systems beyond traditional hardware RAID and ZFS; there are quite a few interesting ideas out there you should open your mind to. Btrfs would be a great place to start your investigation.
Dylan16807 5y ago
If 8 drives is safe with dual parity, then significantly more than 8 drives would be safe with triple parity, right?
- Dylan16807 5y ago
  I did some calculations, with an estimated 3% annual failure rate per drive. The numbers shift a bit depending on how much you expect drive failures to correlate and how long you take to replace them. So I analyzed two example cases, using pretty good math based on a binomial distribution. The first case is a pessimistic one, where after a drive fails we assume that any other possible failures from the next year will hit before we finish rebuilding. In this case, 8 drives with double parity have a 1/740 chance of array failure each year. If we switch to triple parity, we can have 16 drives and a slight improvement in safety. The second case is a more optimistic one, where after a drive fails we assume that any other possible failures from the next week will hit before we finish rebuilding. In this case, 8 drives with double parity have a 1 / 93 million chance of array failure each week. If we switch to triple parity, we can have 40 drives and a slight improvement in safety. I would have done "what if all the drive failures happened at once" but that gives >10% odds of an 8 drive array failing, which seems unreasonably harsh.