3 ms·

There are two nasty thing going on with raid5: a) You can tolerate 2 drive failures, no more. One less, but if you tolerated one, you can tolerate one more, and

by tetha 8y ago

There are two nasty thing going on with raid5: a) You can tolerate 2 drive failures, no more. One less, but if you tolerated one, you can tolerate one more, and no more. AND b) raid rebuild causes MASSIVE stress on the remaining drives. Seriously massive stress, beyond what distributed systems or raid 1 / 6 based systems do. This occurs because RAID5 both rewrites parities, but it also has to re-read data from all drives while writing parities to all drives. That's a lot of random access and especially spin-drives and larger drives (see the coincidence?) dislike that. That tends to cause similar-aged drives to die and then your raid is gone. I'd suppose this is less hard on SSDs than HDDs. But there's still a lot of rewrites going on, and SSDs don't like that either.

rincebrain 8y ago
a) RAID5 can fault precisely one drive without data loss. Two overlapping errors (e.g. two errors in one stripe) and you're up shit creek without a paddle. This is the generic definition for RAID5, not an implementation detail. b) What makes you think RAID6 doesn't also incur this? The only difference is that RAID6 also includes a Q parity block in each stripe, so the only thing you get saved from is if you don't need to read the parity on the stripes, you save 1/(N+P-1) IOs per drive. RAID6 is still going to need to recompute ~2/(N+P) parities (one P, and one Q) for rebuilding a drive over (N+P) stripes; and reconstruct the data for the rest (depending on how P and Q are implemented, they could interweave which they use for reconstructions, but AIUI it's generally more expensive to recompute from Q than P, and R than P or Q, in many instances of this math). c) Many RAID systems can rebuild starting from the starts of the respective disks and streaming along (or, in recent ZFS's case, coalescing the IOs to be in sequential order groups and issuing them), though certainly not all of them. The logic "usually" goes that RAID5/RAID6 rebuilds are dangerous because they involve reading all the bits, so to speak, so if you don't have an equivalent of scheduled patrol reads to be sure bits at rest that haven't been read by users haven't gone south, you'll first discover this...during a rebuild, and with RAID5, you're SOL.

by tetha 8y ago

rincebrain 8y ago

a) RAID5 can fault precisely one drive without data loss. Two overlapping errors (e.g. two errors in one stripe) and you're up shit creek without a paddle. This is the generic definition for RAID5, not an implementation detail. b) What makes you think RAID6 doesn't also incur this? The only difference is that RAID6 also includes a Q parity block in each stripe, so the only thing you get saved from is if you don't need to read the parity on the stripes, you save 1/(N+P-1) IOs per drive. RAID6 is still going to need to recompute ~2/(N+P) parities (one P, and one Q) for rebuilding a drive over (N+P) stripes; and reconstruct the data for the rest (depending on how P and Q are implemented, they could interweave which they use for reconstructions, but AIUI it's generally more expensive to recompute from Q than P, and R than P or Q, in many instances of this math). c) Many RAID systems can rebuild starting from the starts of the respective disks and streaming along (or, in recent ZFS's case, coalescing the IOs to be in sequential order groups and issuing them), though certainly not all of them. The logic "usually" goes that RAID5/RAID6 rebuilds are dangerous because they involve reading all the bits, so to speak, so if you don't have an equivalent of scheduled patrol reads to be sure bits at rest that haven't been read by users haven't gone south, you'll first discover this...during a rebuild, and with RAID5, you're SOL.