3 ms·

An HDD/SSD without ECC will not work, and wouldn't have worked for a long time now. Many of the reads performed will require some ECC use. ECC is always writte

by baruch 10y ago

An HDD/SSD without ECC will not work, and wouldn't have worked for a long time now. Many of the reads performed will require some ECC use. ECC is always written alongside the actual block and the overhead for ECC is the reason for the move from 512b sectors to 4Kb sectors in HDDs. For SSDs the data is already written in different block sizes depending on the NAND and the internal representation and ECC is done for larger than 512b units. The probability of failure during rebuild is not really directly linked to drive size, the usual interpretation of the drive BER is wrong (media BER is stated across a large population of drives rather than just one drive).

cmurf 10y ago
Yes, the two key things: population of drives (model/version possibly, but ultimately not publicly known what the sample size is), and the rate is generally a maximum. i.e. use of "less than" symbol with the rate. Therefore, it's wrong to say every x bytes you should expect an unrecoverable read error. We have no idea how good the drives actually are, the reporting is in orders of magnitude anyway so it could be 8 or 9 times more reliable than the reported rate; or even multiple orders of magnitude. It's not untrue to say < 1 URE in 10^14 bits, but for the population to actually experience 1 URE in 10^16 bits. Why not advertise that? Well, maybe there's a product that costs a little more than advertises 1 URE in 10^15 bits. And other product classes that promise better.
cbsmith 10y ago
> An HDD/SSD without ECC will not work, and wouldn't have worked for a long time now. Many of the reads performed will require some ECC use. Yeah, I expressed that badly. I always forget about the ECC bytes that are in the firmware. > The probability of failure during rebuild is not really directly linked to drive size, the usual interpretation of the drive BER is wrong (media BER is stated across a large population of drives rather than just one drive). Regardless of interpretations of BER, I can't agree about drive sizes. The phenomenon of failures during rebuild is well documented and the driving principle behind double-parity RAID. Adam Leventhal (who ought to know this stuff better than either of us) wrote a paper several years back on the need for triple-parity RAID, and it was entirely driven by increased drive densities: http://queue.acm.org/detail.cfm?id=1670144 http://queue.acm.org/detail.cfm?id=1670144 The reality is the higher drive densities mean you lose more bytes at a time when you have a drive failure, and that means more bytes you want to have "recovered".
- baruch 10y ago
  Adam Leventhal uses the wrong interpretation of the BER value so I wouldn't take his words at face value. The whole argument of Adam is by the disk BER and not by density. I do assume though that the RAID array does media scrub (BMS) periodically, if you don't you are at risk anyway and I'd call that negligence in maintaining your RAID. If you do use scrubbing the risk that another drive has a bad media spot is low as it must have developed in the time from the last scan and that is a bounded time (week, two, four) so the risk of two drives having a bad sector is now even lower (though never non-zero, backups are still a thing). If you couple that also with TLER and proper media handling instead of dropping the disk on media error the risk to the data becomes very low since there isn't a very high likelyhood that two disks will have a bad sector in the same stripe. I've been working with HDDs and SSDs and developing software for enterprise storage systems for a number of years now, I've worked in XIV and for all the thousands of systems and hundred of thousand disks of many models I've never seen two disks fail to read the same stripe and RAID recovery was always possible. Other problems are more likely sooner than an actual RAID failure (technician shutting down the system by pressing the UPS buttons or a software bug). I did learn of several failure modes that can increase the risk but they depend on specific workloads that are not generally applicable. One of those is that if you write to one track all too often you may affect nearby tracks and if the workload is high enough you don't give the disk the time to fix this (the disk tracks this failure mode and will workaround it in idle time). In such a case same stripe can be affected in multiple disks and the time to develop may (or may not) be shorter than the media scrub time. And even then a thin provisioned raid structure would reduce the risk of this failure mode and giving disks some rest time (even just a few seconds) would allow the drive to fix this and other problems it knows about. All in all, RAID is not dead (yet).
  - cbsmith 10y ago
    So why would one ever bother with RAID-6 (or its moral equivalent with RAID-Z)? I've yet to hear someone justify it from a requirement being to actually survive two drives failing simultaneously.
    - 10y ago

by baruch 10y ago

cmurf 10y ago

Yes, the two key things: population of drives (model/version possibly, but ultimately not publicly known what the sample size is), and the rate is generally a maximum. i.e. use of "less than" symbol with the rate. Therefore, it's wrong to say every x bytes you should expect an unrecoverable read error. We have no idea how good the drives actually are, the reporting is in orders of magnitude anyway so it could be 8 or 9 times more reliable than the reported rate; or even multiple orders of magnitude. It's not untrue to say < 1 URE in 10^14 bits, but for the population to actually experience 1 URE in 10^16 bits. Why not advertise that? Well, maybe there's a product that costs a little more than advertises 1 URE in 10^15 bits. And other product classes that promise better.

cbsmith 10y ago

> An HDD/SSD without ECC will not work, and wouldn't have worked for a long time now. Many of the reads performed will require some ECC use. Yeah, I expressed that badly. I always forget about the ECC bytes that are in the firmware. > The probability of failure during rebuild is not really directly linked to drive size, the usual interpretation of the drive BER is wrong (media BER is stated across a large population of drives rather than just one drive). Regardless of interpretations of BER, I can't agree about drive sizes. The phenomenon of failures during rebuild is well documented and the driving principle behind double-parity RAID. Adam Leventhal (who ought to know this stuff better than either of us) wrote a paper several years back on the need for triple-parity RAID, and it was entirely driven by increased drive densities: http://queue.acm.org/detail.cfm?id=1670144 http://queue.acm.org/detail.cfm?id=1670144 The reality is the higher drive densities mean you lose more bytes at a time when you have a drive failure, and that means more bytes you want to have "recovered".

baruch 10y ago
Adam Leventhal uses the wrong interpretation of the BER value so I wouldn't take his words at face value. The whole argument of Adam is by the disk BER and not by density. I do assume though that the RAID array does media scrub (BMS) periodically, if you don't you are at risk anyway and I'd call that negligence in maintaining your RAID. If you do use scrubbing the risk that another drive has a bad media spot is low as it must have developed in the time from the last scan and that is a bounded time (week, two, four) so the risk of two drives having a bad sector is now even lower (though never non-zero, backups are still a thing). If you couple that also with TLER and proper media handling instead of dropping the disk on media error the risk to the data becomes very low since there isn't a very high likelyhood that two disks will have a bad sector in the same stripe. I've been working with HDDs and SSDs and developing software for enterprise storage systems for a number of years now, I've worked in XIV and for all the thousands of systems and hundred of thousand disks of many models I've never seen two disks fail to read the same stripe and RAID recovery was always possible. Other problems are more likely sooner than an actual RAID failure (technician shutting down the system by pressing the UPS buttons or a software bug). I did learn of several failure modes that can increase the risk but they depend on specific workloads that are not generally applicable. One of those is that if you write to one track all too often you may affect nearby tracks and if the workload is high enough you don't give the disk the time to fix this (the disk tracks this failure mode and will workaround it in idle time). In such a case same stripe can be affected in multiple disks and the time to develop may (or may not) be shorter than the media scrub time. And even then a thin provisioned raid structure would reduce the risk of this failure mode and giving disks some rest time (even just a few seconds) would allow the drive to fix this and other problems it knows about. All in all, RAID is not dead (yet).
- cbsmith 10y ago
  So why would one ever bother with RAID-6 (or its moral equivalent with RAID-Z)? I've yet to hear someone justify it from a requirement being to actually survive two drives failing simultaneously.
  - 10y ago