4 ms·

>Only if you use two drives per mirror; RAID10's worst-case matches RAID6's when using three disks per mirror. That's not a proper comparison as you are now us

by pedrocr 11y ago

>Only if you use two drives per mirror; RAID10's worst-case matches RAID6's when using three disks per mirror. That's not a proper comparison as you are now using more disks in the RAID10 array than the RAID6 to get the same space. If you're willing to have 6 disks to get the space of 2 you could use an array with 3 parity disks (what ZFS calls Z3) and now you can fail any 3 disks just like RAID10 and you've only used 5 disks instead of 6. If you have the option for 4 parity disks (I don't think ZFS provides a Z4 level) you could fail any 4 drives of 6 and get the space of 2, so 1 better than RAID10. N-Parity setups are strictly more redundant than RAID10 for the same number of disks and required capacity. >actually mitigates the "oh no I bought a bad batch of hard drives and now my RAID is kaput" failure case (unlike with RAID6, where mixing/matching vendors won't help you unless you mix/match them for every disk). This makes no sense to me. I mix and match vendors in RAID6 arrays just fine and it lowers the correlation of failures between disks just as well. In fact parity RAID is again strictly better at this as well. Since the redundancy is across the whole array instead of split into clusters whenever you add more drives from different manufacturers you're adding that redundancy to the whole array. With RAID10 if you have 3 drives per mirror your maximum redundancy before losing data is 3 manufacturers. >There are also some recoverability benefits of RAID10 v. RAID6; notably, recovering from a fault in RAID6 requires recomputing parity, while no such thing is necessary for RAID10. This mitigates the "oh no my array took too long to fix itself and now my RAID is kaput" failure case. Yeah, performance is the real advantage of RAID10 as I mentioned and indeed that performance can be important during rebuilds so it has some durability implications.

yellowapple 11y ago
> That's not a proper comparison as you are now using more disks in the RAID10 array than the RAID6 to get the same space. You already have to use more disks for a RAID10 than in a RAID6, so I don't see why this is that big of a deal. Anything involving mirroring will involve at least a 50% cut in capacity (or more, such as with a 3-way or even 4-way mirror). RAID1 and RAID10 don't prioritize capacity; they prioritize robustness. > N-Parity setups are strictly more redundant than RAID10 for the same number of disks and required capacity. But not more robust against failure, as I've already described (hopefully) rather clearly. If we want to play this game, I can keep adding more disks per mirror, and now my worst-case will match that of your n-parity array while having a vastly better best-case (and, additionally, dragging the probability of that worst-case happening closer and closer to zero). Remember: if you're a RAID1 or RAID10 user who cares about disk capacity, you're doing something wrong. > I mix and match vendors in RAID6 arrays just fine and it lowers the correlation of failures between disks just as well. Do you mix and match all disks, though? Because if you have at least three (in the case of RAID6) disks, no matter where they are in the array, that have the same failure curve (i.e. are the same model/manufacturer), then you're gonna have a bad time when the first one of them dies. That's my point. In order to get the same inherent benefit as RAID10 here, you'd have to have a different vendor/model for every single drive in your RAID6 array. > whenever you add more drives from different manufacturers you're adding that redundancy to the whole array. Only if they're all different. You'll eventually get to the point where there aren't enough manufacturers in the world (or you'll be compromising on manufacturer diversity - which is, granted, what most people do). > With RAID10 if you have 3 drives per mirror your maximum redundancy before losing data is 3 manufacturers. And that's fine, because if one of those manufacturers sold me a defective batch, the problem's isolated to that side of the mirror. That's my point; with RAID10, it's no longer a game of sheer numbers of different models and manufacturers, but instead the more manageable strategy of giving each side of the mirror a different bathtub curve to ride, thus mitigating the chance of multiple hard drive batches failing at the same time.
- pedrocr 11y ago
  > You already have to use more disks for a RAID10 than in a RAID6, so I don't see why this is that big of a deal. Not really. RAID6 and RAID10 are only directly comparable at 4 disks as they have the same capacity with the same number of disks. And in that case RAID6 is strictly better than RAID10 (any 2 disks can fail vs at best 2 disks can fail). >But not more robust against failure, as I've already described (hopefully) rather clearly. If we want to play this game, I can keep adding more disks per mirror, and now my worst-case will match that of your n-parity array while having a vastly better best-case This is simply not true. An n-parity array will always be strictly (and I mean strictly) more redundant than the equivalent RAID array. For 6 disks this means using 4 parity disks which allows failing any 4 disks which is RAID10's best case but better than it's worst case (any 2 disks can fail). If you go for 8 disks RAID10 allows a best case where 6 disks can fail which with a 6-parity raid is your normal case, and so on. Parity is strictly better than mirroring at redundancy its just usually not worth it for performance reasons. At the file level it may make sense though (Backblaze does file level 3-parity for example) as you can choose to take the hit only on certain files. Brtfs also allows per-file raid levels so it's too bad they're not pursuing that external patch for n-parity raid levels that the original article mentions. (skipped some parts as the core of the issue is below) >And that's fine, because if one of those manufacturers sold me a defective batch, the problem's isolated to that side of the mirror. That's my point; with RAID10, it's no longer a game of sheer numbers of different models and manufacturers, but instead the more manageable strategy of giving each side of the mirror a different bathtub curve to ride, thus mitigating the chance of multiple hard drive batches failing at the same time. This is not a RAID10 advantage it's a way to mitigate the RAID10 disadvantage compared to parity. Parity RAID with the same capacity/number of disks can survive more disk failures than RAID10 (as I explained above). In both cases you want the pool of N disks to be as diversified as possible to avoid correlation between failures. Parity exploits that lack of correlation completely (any disk failure is just like any other, it doesn't matter which disk is which) whereas RAID10 will blow up earlier if all disks in one side fail at once so you mitigate that by making the two sides different.
  - yellowapple 11y ago
    > RAID6 and RAID10 are only directly comparable at 4 disks They're really not ever directly comparable, because RAID10 isn't a hard-and-fast RAID level like RAID6, but rather the combination of two (really, one plus a quasi-level, but whatever). RAID10 encapulates any situation where mirrors are striped together. > For 6 disks this means using 4 parity disks which allows failing any 4 disks which is RAID10's best case but better than it's worst case (any 2 disks can fail). You're getting caught up on redundancy for a given total number of disks, and in the process missing the point entirely. If we really want to play that game, then I'll just roll a RAID1 (a.k.a. a RAID10 with only one set of mirrors) and call it a day, because with a n-disk array, a RAID1 will always be at least as redundant as the equivalent parity-based RAID (really, it'll always be able to survive one more disk failure than an n-1 parity-block array; you can see this rather easily when comparing RAID5 with a three-way RAID1). From there, my point should be more clear. Just like how you can constantly add more and more parity blocks (and disks to handle them), I can constantly widen the mirror. Eventually we'll both get to the point where we realize "what the hell are we doing; there's 100 copies of every block; we should start adding some more non-redundancy disks", at which point RAID10's advantages become more clear, since each mirror group added to the striped whole instantly bumps up capacity and the best-case without compromising the worst-case. Meanwhile, you do have the advantage of having more flexibility in the quantity of disks added, but doing so doesn't help your best-case unless you feel like resuming the whole "let's make every disk a parity or mirror disk" arms race. And before you mention it: yes, I know RAID1 isn't traditionally counted under RAID10, but a RAID10 with only one mirror group is possible to configure using tools like `mdadm`, and while this wouldn't look different from a RAID1 at first glance, it would certainly be different as soon as more mirror groups are added. > Parity is strictly better than mirroring at redundancy I think this remark is the core of your misunderstanding here. Are you really trying to claim than an n-sized parity array with n-1 parity blocks is better than a RAID1? You might want to clarify your position there if that's not what you meant to suggest :) > Parity exploits that lack of correlation completely (any disk failure is just like any other, it doesn't matter which disk is which) Which is exactly why you're incorrect. Since any disk failure is like any other, every single disk has to be unique in order for a parity-based array to benefit from any diversity. With mirroring, you don't have to go nearly as far for that same benefit; you just need each mirror to have a different bathtub curve. That's my point. -- This has been a fun discussion, and there is certainly lots of room for debate on the merits of mirroring v. parity, but I'm starting to think that we should just agree to disagree on this one.

by pedrocr 11y ago

yellowapple 11y ago

> That's not a proper comparison as you are now using more disks in the RAID10 array than the RAID6 to get the same space. You already have to use more disks for a RAID10 than in a RAID6, so I don't see why this is that big of a deal. Anything involving mirroring will involve at least a 50% cut in capacity (or more, such as with a 3-way or even 4-way mirror). RAID1 and RAID10 don't prioritize capacity; they prioritize robustness. > N-Parity setups are strictly more redundant than RAID10 for the same number of disks and required capacity. But not more robust against failure, as I've already described (hopefully) rather clearly. If we want to play this game, I can keep adding more disks per mirror, and now my worst-case will match that of your n-parity array while having a vastly better best-case (and, additionally, dragging the probability of that worst-case happening closer and closer to zero). Remember: if you're a RAID1 or RAID10 user who cares about disk capacity, you're doing something wrong. > I mix and match vendors in RAID6 arrays just fine and it lowers the correlation of failures between disks just as well. Do you mix and match all disks, though? Because if you have at least three (in the case of RAID6) disks, no matter where they are in the array, that have the same failure curve (i.e. are the same model/manufacturer), then you're gonna have a bad time when the first one of them dies. That's my point. In order to get the same inherent benefit as RAID10 here, you'd have to have a different vendor/model for every single drive in your RAID6 array. > whenever you add more drives from different manufacturers you're adding that redundancy to the whole array. Only if they're all different. You'll eventually get to the point where there aren't enough manufacturers in the world (or you'll be compromising on manufacturer diversity - which is, granted, what most people do). > With RAID10 if you have 3 drives per mirror your maximum redundancy before losing data is 3 manufacturers. And that's fine, because if one of those manufacturers sold me a defective batch, the problem's isolated to that side of the mirror. That's my point; with RAID10, it's no longer a game of sheer numbers of different models and manufacturers, but instead the more manageable strategy of giving each side of the mirror a different bathtub curve to ride, thus mitigating the chance of multiple hard drive batches failing at the same time.

pedrocr 11y ago
> You already have to use more disks for a RAID10 than in a RAID6, so I don't see why this is that big of a deal. Not really. RAID6 and RAID10 are only directly comparable at 4 disks as they have the same capacity with the same number of disks. And in that case RAID6 is strictly better than RAID10 (any 2 disks can fail vs at best 2 disks can fail). >But not more robust against failure, as I've already described (hopefully) rather clearly. If we want to play this game, I can keep adding more disks per mirror, and now my worst-case will match that of your n-parity array while having a vastly better best-case This is simply not true. An n-parity array will always be strictly (and I mean strictly) more redundant than the equivalent RAID array. For 6 disks this means using 4 parity disks which allows failing any 4 disks which is RAID10's best case but better than it's worst case (any 2 disks can fail). If you go for 8 disks RAID10 allows a best case where 6 disks can fail which with a 6-parity raid is your normal case, and so on. Parity is strictly better than mirroring at redundancy its just usually not worth it for performance reasons. At the file level it may make sense though (Backblaze does file level 3-parity for example) as you can choose to take the hit only on certain files. Brtfs also allows per-file raid levels so it's too bad they're not pursuing that external patch for n-parity raid levels that the original article mentions. (skipped some parts as the core of the issue is below) >And that's fine, because if one of those manufacturers sold me a defective batch, the problem's isolated to that side of the mirror. That's my point; with RAID10, it's no longer a game of sheer numbers of different models and manufacturers, but instead the more manageable strategy of giving each side of the mirror a different bathtub curve to ride, thus mitigating the chance of multiple hard drive batches failing at the same time. This is not a RAID10 advantage it's a way to mitigate the RAID10 disadvantage compared to parity. Parity RAID with the same capacity/number of disks can survive more disk failures than RAID10 (as I explained above). In both cases you want the pool of N disks to be as diversified as possible to avoid correlation between failures. Parity exploits that lack of correlation completely (any disk failure is just like any other, it doesn't matter which disk is which) whereas RAID10 will blow up earlier if all disks in one side fail at once so you mitigate that by making the two sides different.
- yellowapple 11y ago
  > RAID6 and RAID10 are only directly comparable at 4 disks They're really not ever directly comparable, because RAID10 isn't a hard-and-fast RAID level like RAID6, but rather the combination of two (really, one plus a quasi-level, but whatever). RAID10 encapulates any situation where mirrors are striped together. > For 6 disks this means using 4 parity disks which allows failing any 4 disks which is RAID10's best case but better than it's worst case (any 2 disks can fail). You're getting caught up on redundancy for a given total number of disks, and in the process missing the point entirely. If we really want to play that game, then I'll just roll a RAID1 (a.k.a. a RAID10 with only one set of mirrors) and call it a day, because with a n-disk array, a RAID1 will always be at least as redundant as the equivalent parity-based RAID (really, it'll always be able to survive one more disk failure than an n-1 parity-block array; you can see this rather easily when comparing RAID5 with a three-way RAID1). From there, my point should be more clear. Just like how you can constantly add more and more parity blocks (and disks to handle them), I can constantly widen the mirror. Eventually we'll both get to the point where we realize "what the hell are we doing; there's 100 copies of every block; we should start adding some more non-redundancy disks", at which point RAID10's advantages become more clear, since each mirror group added to the striped whole instantly bumps up capacity and the best-case without compromising the worst-case. Meanwhile, you do have the advantage of having more flexibility in the quantity of disks added, but doing so doesn't help your best-case unless you feel like resuming the whole "let's make every disk a parity or mirror disk" arms race. And before you mention it: yes, I know RAID1 isn't traditionally counted under RAID10, but a RAID10 with only one mirror group is possible to configure using tools like `mdadm`, and while this wouldn't look different from a RAID1 at first glance, it would certainly be different as soon as more mirror groups are added. > Parity is strictly better than mirroring at redundancy I think this remark is the core of your misunderstanding here. Are you really trying to claim than an n-sized parity array with n-1 parity blocks is better than a RAID1? You might want to clarify your position there if that's not what you meant to suggest :) > Parity exploits that lack of correlation completely (any disk failure is just like any other, it doesn't matter which disk is which) Which is exactly why you're incorrect. Since any disk failure is like any other, every single disk has to be unique in order for a parity-based array to benefit from any diversity. With mirroring, you don't have to go nearly as far for that same benefit; you just need each mirror to have a different bathtub curve. That's my point. -- This has been a fun discussion, and there is certainly lots of room for debate on the merits of mirroring v. parity, but I'm starting to think that we should just agree to disagree on this one.