4 ms·

Do you know some real case where ECC memory brought tangible benefits? It seems conventional memory chips are quite reliable.

by effie 11y ago

ggreer 11y ago
Bit errors have been used to hijack communications. Just register a bunch of domains that differ from popular ones by a single bit and watch the requests come in.[1] Typically, users don't recognize single bit errors. Something breaks and they think, "Stupid buggy software." With modern RAM capacities, you're pretty likely to get some bits flipped every few months. 1. http://dinaburg.org/bitsquatting.html http://dinaburg.org/bitsquatting.html
- comboy 11y ago
  > Typically, users don't recognize single bit errors. Something breaks and they think, "Stupid buggy software." With modern RAM capacities, you're pretty likely to get some bits flipped every few months. Supposedly much higher than that[1]. More like "5 single bit errors in 8 Gigabytes of RAM per hour using the top-end error rate" [2] 1. http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf 2. https://en.wikipedia.org/wiki/ECC_memory https://en.wikipedia.org/wiki/ECC_memory
harshreality 11y ago
Memory bit flips happen all the time—not like daily or weekly per computer, but you can have multiple memory bit flip incidents during the multi-year life of an average computer. Either you detect them, or you get silent data corruption. If you're lucky it's in unused ram; if you're not, it may be in running code (possible crash or software misbehavior) or data (in a text document, one corrupted character; in some other kind of data, it could be much worse). Think about how a bit flip in code or data can affect things like ciphertext generation. It can become a huge deal. Think of "bad ram" —the kind that fails memtest86— as ram that has an increased tendency (compared to "good ram") to flip certain bits that with a little more luck, or more conservative timings, wouldn't flip. The threshold between good ram and bad ram is not black and white. Also see "rowhammer" — https://news.ycombinator.com/item?id=9171722 https://news.ycombinator.com/item?id=9171722 — ECC mitigates that to a significant degree. Aggressive memory testing before use can weed out memory that's plain bad (one bit always screwing up), and certain borderline cases of memory that will flip bits under certain high stress loads or access patterns (I suppose that's due to thermal entropy?), but it's no guarantee, and it certainly offers no protection against cosmic ray induced bit flips, which happen too. Ordinary consumer memory is not "quite reliable" if you care about data integrity or stability. Google's study: http://research.google.com/pubs/pub35162.html http://research.google.com/pubs/pub35162.html (direct pdf link: http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/35162.pdf http://static.googleusercontent.com/media/research.google.co... )
- sliverstorm 11y ago
  You're spinning the odds a bit. The gross majority of data in memory is never written to persistent storage. If you're a typical consumer, then of that data that is written, only a tiny fraction is user data, and of that user data, media usually represents most of the volume. Media is tolerant to a random bit flip. Enterprise needs to worry about these things due to sheer scale and things like automatic propagation, but the odds of getting a bit flip in your tax spreadsheet or research paper and then writing it to disk are incredibly small, and personally I figure versioning/backups protect me from a significant portion of that (small) risk. Which brings up disk crashes. Not to divert the conversation, but disk crashes are massively more dangerous than bit flips, and so many people haven't taken even the first steps against them...
toast0 11y ago
At work, we see ECC events frequently (our servers are usually at least 128 GB, so we have higher chances). Most of which are correctable. Some servers will register a couple of correctable events a week, which I would count as a tangible benefit. If not for ecc, this servers would have corrupted data, possibly undetected for a long time, or possibly resulting in unexplained poor reliability; once we figured it out, we would need to spend time figuring out which ram stick is malfunctioning, too, instead of using the diagnostic less on the motherboard. Many times the ECC errors come in a huge batch, and the resulting NMIs make the server unusable, but again we have diagnostics, and not unexplained failures.
noipv4 11y ago
You really have to read the wikipedia page on ECC. There is the result of a large scale study by Google. Beside that, if you use a workstation with ECC memory and regularly utilized a good portion of that RAM (I use 128GB for bioinformatics / genomics applications), there is a good chance of witnessing an ECC error / MCE (Machine check exception). Here's the tl;dr "(i.e. about 5 single bit errors in 8 Gigabytes of RAM per hour using the top-end error rate), and more than 8% of DIMM memory modules affected by errors per year."
tenfingers 11y ago
On a small data center we have, running 6 V40z servers with 64gb of memory each, in a controlled environment at 25c, we register roughly one parity event a month per-system. And that's ~6-7 years old memory, which is less dense than modern chips, in a ideal temperature environment (the racks are water-cooled), down in a basement. You average laptop is never going to register less than that, considering you're carrying it around with less environmental shielding, overheat it, and so on. In fact, with a laptop with 64gb of memory that has gbs of paged data in memory for weeks, the chances are greater that your laptop with linux will crash due to a random bitflip that goes unnoticed, as opposed to a kernel bug. And there's simply no way to tell. How about that?

Do you know some real case where ECC memory brought tangible benefits? It seems conventional memory chips are quite reliable.

by effie 11y ago

Do you know some real case where ECC memory brought tangible benefits? It seems conventional memory chips are quite reliable.

ggreer 11y ago

Bit errors have been used to hijack communications. Just register a bunch of domains that differ from popular ones by a single bit and watch the requests come in.[1] Typically, users don't recognize single bit errors. Something breaks and they think, "Stupid buggy software." With modern RAM capacities, you're pretty likely to get some bits flipped every few months. 1. http://dinaburg.org/bitsquatting.html http://dinaburg.org/bitsquatting.html

comboy 11y ago
> Typically, users don't recognize single bit errors. Something breaks and they think, "Stupid buggy software." With modern RAM capacities, you're pretty likely to get some bits flipped every few months. Supposedly much higher than that[1]. More like "5 single bit errors in 8 Gigabytes of RAM per hour using the top-end error rate" [2] 1. http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf 2. https://en.wikipedia.org/wiki/ECC_memory https://en.wikipedia.org/wiki/ECC_memory

harshreality 11y ago

Memory bit flips happen all the time—not like daily or weekly per computer, but you can have multiple memory bit flip incidents during the multi-year life of an average computer. Either you detect them, or you get silent data corruption. If you're lucky it's in unused ram; if you're not, it may be in running code (possible crash or software misbehavior) or data (in a text document, one corrupted character; in some other kind of data, it could be much worse). Think about how a bit flip in code or data can affect things like ciphertext generation. It can become a huge deal. Think of "bad ram" —the kind that fails memtest86— as ram that has an increased tendency (compared to "good ram") to flip certain bits that with a little more luck, or more conservative timings, wouldn't flip. The threshold between good ram and bad ram is not black and white. Also see "rowhammer" — https://news.ycombinator.com/item?id=9171722 https://news.ycombinator.com/item?id=9171722 — ECC mitigates that to a significant degree. Aggressive memory testing before use can weed out memory that's plain bad (one bit always screwing up), and certain borderline cases of memory that will flip bits under certain high stress loads or access patterns (I suppose that's due to thermal entropy?), but it's no guarantee, and it certainly offers no protection against cosmic ray induced bit flips, which happen too. Ordinary consumer memory is not "quite reliable" if you care about data integrity or stability. Google's study: http://research.google.com/pubs/pub35162.html http://research.google.com/pubs/pub35162.html (direct pdf link: http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/35162.pdf http://static.googleusercontent.com/media/research.google.co... )

sliverstorm 11y ago
You're spinning the odds a bit. The gross majority of data in memory is never written to persistent storage. If you're a typical consumer, then of that data that is written, only a tiny fraction is user data, and of that user data, media usually represents most of the volume. Media is tolerant to a random bit flip. Enterprise needs to worry about these things due to sheer scale and things like automatic propagation, but the odds of getting a bit flip in your tax spreadsheet or research paper and then writing it to disk are incredibly small, and personally I figure versioning/backups protect me from a significant portion of that (small) risk. Which brings up disk crashes. Not to divert the conversation, but disk crashes are massively more dangerous than bit flips, and so many people haven't taken even the first steps against them...

toast0 11y ago

At work, we see ECC events frequently (our servers are usually at least 128 GB, so we have higher chances). Most of which are correctable. Some servers will register a couple of correctable events a week, which I would count as a tangible benefit. If not for ecc, this servers would have corrupted data, possibly undetected for a long time, or possibly resulting in unexplained poor reliability; once we figured it out, we would need to spend time figuring out which ram stick is malfunctioning, too, instead of using the diagnostic less on the motherboard. Many times the ECC errors come in a huge batch, and the resulting NMIs make the server unusable, but again we have diagnostics, and not unexplained failures.

noipv4 11y ago

You really have to read the wikipedia page on ECC. There is the result of a large scale study by Google. Beside that, if you use a workstation with ECC memory and regularly utilized a good portion of that RAM (I use 128GB for bioinformatics / genomics applications), there is a good chance of witnessing an ECC error / MCE (Machine check exception). Here's the tl;dr "(i.e. about 5 single bit errors in 8 Gigabytes of RAM per hour using the top-end error rate), and more than 8% of DIMM memory modules affected by errors per year."

tenfingers 11y ago

On a small data center we have, running 6 V40z servers with 64gb of memory each, in a controlled environment at 25c, we register roughly one parity event a month per-system. And that's ~6-7 years old memory, which is less dense than modern chips, in a ideal temperature environment (the racks are water-cooled), down in a basement. You average laptop is never going to register less than that, considering you're carrying it around with less environmental shielding, overheat it, and so on. In fact, with a laptop with 64gb of memory that has gbs of paged data in memory for weeks, the chances are greater that your laptop with linux will crash due to a random bitflip that goes unnoticed, as opposed to a kernel bug. And there's simply no way to tell. How about that?