5 ms·
Ah, the RAID array love story: one of many similar disks fails, you replace it, array starts rebuilding which causes an increase of reads from other drives and
by voytec 3y ago
Ah, the RAID array love story: one of many similar disks fails, you replace it, array starts rebuilding which causes an increase of reads from other drives and before rebuild is complete, more disks from the same batch start failing.
I don't deal with storage hardware much for some 6+ years, since I've switched to cloudy solutions. But at the beginning of my shared storage building journey, I considered arrays built from different disks somewhat "lame" or "unprofessional". I changed this viewpoint 180° after seeing multiple similar drives fail in a very short (hours, days) timespan for the second time.
I now have more trust for home-built ZFS setup using different disks than a dedicated storage appliance using enterprise-grade disks with almost following serial numbers.
- katbyte 3y agoRaid array should always be scrubbed every few months to prevent this exact situation
- nabeards 3y agoScrubbed as in erased? Where do you temporarily place 1PB+ of data while that’s happening?
- bell-cot 3y ago`zpool scrub $Name_Of_zfs_Storage_Pool` https://docs.oracle.com/cd/E18752_01/html/819-5461/gbbwa.html https://docs.oracle.com/cd/E18752_01/html/819-5461/gbbwa.htm...
- duskwuff 3y agoScrubbed, as in performing a periodic read of the entire array to 1) check the data for consistency and 2) exercise the drives.
- deleted 3y ago[deleted]
- singron 3y agoScrubbing helps fix random errors in the drive data, but it won't prevent correlated disk failure. If a drive fails during the scrub, then you still have to replace the disk, and it's still very likely that another disk can fail in that time. If you just want to eliminate lemons, then you can do a burn-in test before adding it to your array, but IMO even bad disks usually don't fail that quickly.