3 ms·
The more interesting data point is that the "URE" spec. That is "unrecoverable read error". All of these drives do track at a time with full ECC to facilitate r
by ChuckMcM 6y ago
The more interesting data point is that the "URE" spec. That is "unrecoverable read error". All of these drives do track at a time with full ECC to facilitate reading in the presence of noise, but a track write can have effects on adjacent tracks and temperature can affect head flying height, and vibration can affect head tracking, etc. All of which, statistically, can create a non-recoverable bit error. Typically this spec is like 1 in 10^14 bits.
What it interesting about this spec is that it is statistical, meaning that the more times you read the bits, the more likely you are to hit a URE. HDD manufacturers do all sorts of things to avoid them (re-trys, track mirroring, etc) but ultimately it is a mechanical system and there are a LOT of bits.
When I was at NetApp, the company would get reports of UREs that were fixed by RAID reconstruction (so you read a disk, you get an error and you fix it by the ECC in the other drives). And we had determined that by 4TB it would be "stupid" to using mirroring for protection since if a drive failed you couldn't count on being able to re-silver the mirror by getting a clean read of the other half. (something like 1 in 20 attempts would fail and you'd have data loss). And of course reconstructing a RAID-5 group for a failed drive you need to read the other drives, for larger groups of disks (which people did to maximize data storage) you started becoming at risk of being able to reconstruct all of the stripes. That was the genesis of doing dual parity (which Netapp called 'diagonal parity' because of the design).
The ZFS folks have also considered this and designed both dual and TRIPLE parity which makes sense for these larger drives!
It already takes a long time to reconstruct any "wide" vDev in ZFS that is zraid2. So one wonders if you could solve for device capacity where 'reconstruction takes 3 years' (the depreciated life span of a drive at Google).
- NateLawson 6y agoKirk McKusick took this kind of scale into consideration when designing softupdates for UFS. One of his criticisms of the competing log structured filesystems at the time was that the replay process would require RAM in excess of any server's capacity once disk drives were large enough. He had designed the softupdates recovery process to be incremental and use a fixed amount of RAM that didn't scale with volume size. This kind of holistic systems' awareness is a sign of software that's designed well.
- effie 6y agoAgreed on 4TB to be a reasonable upper limit for RAID1 if URE rate is 1e-14. However, why would 1 in 20 rebuilds fail? And if it did, why would there be a data loss? The data is OK as long as the remaining disk is alive, you can retry the rebuild.