5 ms·
ECC works if done right. Accessing a memory location can fix bit-flips (ECC is a 'correcting' code). But systems that don't regularly visit every memory locatio
by JoeAltmaier 6y ago
ECC works if done right. Accessing a memory location can fix bit-flips (ECC is a 'correcting' code). But systems that don't regularly visit every memory location, can accumulate risk. Those dark corners of RAM can eventually get double-bit errors and be uncorrectable. So an OS might 'wash' RAM during idle moments, reading every location in a round-robin manner to get ECC to kick in and auto-correct. Doesn't matter how fast (1M every hour or whatever) as long as somehow ECC has a chance to work.
- jacquesm 6y agoInteresting, similar to scrubbing raid arrays. How often do those double bitflips appear though? You'd have to have a pretty long running server for that to be a problem, no?
- jeffbee 6y agoAccording to Google's old paper on the subject, about 1% of their machines suffered from an uncorrectable (i.e. multi-bit) error in a year.
- musingsole 6y agoA double-bit error in many cases is fine. If the error is at least detectable at the time of a read, your protection worked. What's scary is a triple-flip event. Most of those will still look like corrupted data, but if it happens to flip into looking like a fixable, single-bit error, you're out of luck and won't even know it.
- a1369209993 6y ago> Most of those will still look like corrupted data, Not if you're using a typical 72-bit SECDED code[0]. You have two error indicators: a summary parity bit (even number of errors: 0,2,etc vs odd number of errors: 1,etc), and a error index: 0 for no errors, or the bitwise xor of the locations each bit error. For a triple error at bits a,b, and c, you'll have summary parity of 1 (odd number of errors, assumed to be 1), and a error index of a^b^c, in the range 0..127, of which 0..71[1] (56.25%, a clear albeit not overwhelming majority) will correspond to legitimate single-bit errors. 0: https://en.wikipedia.org/wiki/Hamming_code#Hamming_codes_with_additional_parity_(SECDED) https://en.wikipedia.org/wiki/Hamming_code#Hamming_codes_wit... 1: or 72 out of 128 anyway; the active bits might not all be assigned contiguous indexes starting from zero, but it doesn't change the probability and it's simpler to analyse if summary is bit 0 and index bit i is substrate bit 2^i.
- a1369209993 6y ago> but it doesn't change the probability Actually, a,b,c are also sampled from the range of valid bit indexes (not uniformly on 0..127), so you might be able to pick a cardinality-72 subset of 0..127 such that random a^b^c is disproportionately likely to fall outside that subset (and thus get diagnosed as not a valid single-bit error correction). I don't know that any existing ECC implementations actually do that, though. Edit: did some cursory testing and using indexes 0..71 actually catches only ~24.04% (86016/357840) of triple-bit errors, compared the theoretical 43.75% (156555/357840, I think?) from a random error index. So "doesn't change the probability" is just completely wrong given that a,b,c are randomly chosen from the 72 substrate bits, rather than from 128 possible 7-bit indexes. Oddly enough, testing random selections of 72 valid indexes (eg 74773982'EBD0D35C'C5BEB2D8'C3FE9C5E, where set bits correspond to used indexes) actually gives slightly better results than theory (44.97% for that one, 44.65% is the lowest in the last dozen or so), which is somewhat interesting, but I still haven't found any bit assignment that gives better than 50% (178920/357840) catchment of triple bit errors.
- electricshampo1 6y agoPatrol scrub is basically this (https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/xeon-e7-family-ras-server-paper.pdf); https://www.intel.com/content/dam/www/public/us/en/documents... it is built into the memory controller, no OS involvement is needed.
- electricshampo1 6y agoworking link: https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/xeon-e7-family-ras-server-paper.pdf https://www.intel.com/content/dam/www/public/us/en/documents...
- temac 6y agoThe RAM already needs to be refreshed and IIRC it is done by the memory controller when not in sleep mode. However I don't remember if there are provisions for ECC checking in case there are some dedicated refresh commands. I hope so, but I'm not sure.