6 ms·
Translation: "ZFS and btrfs are 'incomplete' because their handling of striped RAID is incomplete." Disregarding the fact that things like mdadm still exist, a
by yellowapple 11y ago
Translation: "ZFS and btrfs are 'incomplete' because their handling of striped RAID is incomplete."
Disregarding the fact that things like mdadm still exist, and further disregarding the fact that the vast majority of filesystems out there don't bother with trying to implement RAID at all (probably because - again - there are things like mdadm that do that already), RAID5/6 are generally a bad idea compared to a RAID1 or RAID10. Both RAID5 and RAID6 make multiple very incorrect assumptions:
* Failure of multiple drives in a short period of time is rare (in reality, if one drive fails (excluding bathtub-curve-related infant mortality), the likelihood of subsequent drive failures increases significantly)
* The failed drive can be replaced and repopulated quickly (in reality, this is becoming less and less true as drives get bigger and bigger, thus taking more and more time to rebuild the failed array member; SSDs buy some time here, but that's not sustainable)
* Bit rot / "cosmic rays" are rare (in reality, silent data errors happen all the time, as has been demonstrated [0] as an example of why RAID5/6 is woefully insufficient for even the most basic protection against data corruption)
Basically, if you want something striping, and care at all about data integrity, go with RAID10. Only use RAID5/6 if you don't care about data loss (whether due to a comprehensive backup policy or a comprehensive redundancy policy on a machine-level), though in that case you might as well just cut to the chase and use RAID0.
I really wish this article would've dug into some of the real shortcomings of these filesystems; btrfs in particular would be incredibly useful compared to a more traditional LVM approach if it supported file encryption and swap subvolumes (for either of these things, LVM (+ LUKS for encryption) is necessary with or without btrfs). Instead they get criticized over not supporting things that a sane sysadmin wouldn't touch with a ten-foot pole in this day and age.
[0]: http://www.miracleas.com/BAARF/Why_RAID5_is_bad_news.pdf http://www.miracleas.com/BAARF/Why_RAID5_is_bad_news.pdf
- stephengillie 11y agoNot to mention the "write hole" where a single write requires 4 reads on RAID5. RAID0+1 provides better write speeds because less compute and fewer reads are needed. Striped RAIDs are a thing of the past. The closest we could come today is, effectively, "spanning datacenters".
- yellowapple 11y agoWorth mentioning that RAID1+0 and RAID0+1 are two different things (striped mirrors versus mirrored stripes, respectively), and have two different probabilities of multi-drive failure causing total array failure (long story short, RAID10 has a better chance of surviving a multi-disk failure than RAID01 without any sacrifice in capacity or performance). But yes, the performance boost is certainly quite fantastic.
- pedrocr 11y ago>Basically, if you want something striping, and care at all about data integrity, go with RAID10. Only use RAID5/6 if you don't care about data loss (whether due to a comprehensive backup policy or a comprehensive redundancy policy on a machine-level), though in that case you might as well just cut to the chase and use RAID0. Your points are valid but this doesn't follow. RAID10 is strictly worse than RAID6 for redundancy. In a four drive array any two drives can fail with RAID6 whereas with RAID10 2 drives may be enough to blow up half your data. RAID10 is actually a less redundancy but higher performance tradeoff compared to RAID6.
- deleted 11y ago[deleted]
- yellowapple 11y agoI should've clarified that RAID1 is the absolute best if data integrity is your sole concern. Regardless... > RAID10 is strictly worse than RAID6 for redundancy. Only if you use two drives per mirror; RAID10's worst-case matches RAID6's when using three disks per mirror. RAID6 also lacks a "best case" that's distinct from its worst case. You lose three disks, you're hosed, period. You lose three disks in a RAID10 (even one with two disks per mirror), it's much more probable that your data will be intact. You can mitigate this further by using a bigger array (RAID10 total failure probability is affected by array size, unlike RAID6) or by using different vendors for each disk in the mirrors (example: each mirror has one Intel and one Samsung SSD of the same capacity) - which, while having some performance and capacity implications, actually mitigates the "oh no I bought a bad batch of hard drives and now my RAID is kaput" failure case (unlike with RAID6, where mixing/matching vendors won't help you unless you mix/match them for every disk). There are also some recoverability benefits of RAID10 v. RAID6; notably, recovering from a fault in RAID6 requires recomputing parity, while no such thing is necessary for RAID10. This mitigates the "oh no my array took too long to fix itself and now my RAID is kaput" failure case.
- pedrocr 11y ago>Only if you use two drives per mirror; RAID10's worst-case matches RAID6's when using three disks per mirror. That's not a proper comparison as you are now using more disks in the RAID10 array than the RAID6 to get the same space. If you're willing to have 6 disks to get the space of 2 you could use an array with 3 parity disks (what ZFS calls Z3) and now you can fail any 3 disks just like RAID10 and you've only used 5 disks instead of 6. If you have the option for 4 parity disks (I don't think ZFS provides a Z4 level) you could fail any 4 drives of 6 and get the space of 2, so 1 better than RAID10. N-Parity setups are strictly more redundant than RAID10 for the same number of disks and required capacity. >actually mitigates the "oh no I bought a bad batch of hard drives and now my RAID is kaput" failure case (unlike with RAID6, where mixing/matching vendors won't help you unless you mix/match them for every disk). This makes no sense to me. I mix and match vendors in RAID6 arrays just fine and it lowers the correlation of failures between disks just as well. In fact parity RAID is again strictly better at this as well. Since the redundancy is across the whole array instead of split into clusters whenever you add more drives from different manufacturers you're adding that redundancy to the whole array. With RAID10 if you have 3 drives per mirror your maximum redundancy before losing data is 3 manufacturers. >There are also some recoverability benefits of RAID10 v. RAID6; notably, recovering from a fault in RAID6 requires recomputing parity, while no such thing is necessary for RAID10. This mitigates the "oh no my array took too long to fix itself and now my RAID is kaput" failure case. Yeah, performance is the real advantage of RAID10 as I mentioned and indeed that performance can be important during rebuilds so it has some durability implications.