3 ms·

Is there any research or even anecdote from people running lots of machines that indicates how frequently corruption occurs? Suppose I'm doing scientific compu

Is there any research or even anecdote from people running lots of machines that indicates how frequently corruption occurs? Suppose I'm doing scientific computing, is ECC solving a super rare problem, or something that might happen weekly or monthly?

sireat 9y ago
I believe there was data that with 64GB of RAM you can expect at least a single bit error or two a month. Thus ECC would be super helpful to those that expect that most of the 64GB will be filled with bits that shouldn't be changed. This article by a system integrator is a pretty decent data point: https://www.pugetsystems.com/labs/articles/Advantages-of-ECC-Memory-520/ https://www.pugetsystems.com/labs/articles/Advantages-of-ECC...
- smitherfield 9y ago
  >>>I believe there was data that with 64GB of RAM you can expect at least a single bit error or two a month. But this would be if you were using 64GB nonstop for a month, which is quite a bit rarer than just having 64GB. And certain filesystems (ZFS, Btrfs) and most databases have software checksumming functionality to detect bitrot. Not to say ECC isn't useful or multiple layers of protection aren't prudent, just that the problem it solves is something you're unlikely to encounter below datacenter scale.
ljcn 9y ago
Some research yes. To answer your final question directly - more like hourly. Google did a big study in 2009 [0] and found error rates between 2.4E-11 and 6.7E-11 errors/bit-hour, or, in more familiar units: about 1 error per 2-GByte of memory per hour. Obviously most errors go unnoticed, but for anything where random bit flips are unacceptable, ECC is necessary. Note too that error rates go up with memory density, not down, so things might be worse with more recent chips. [0] http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf
- api 9y ago
  > about 1 error per 2-GByte of memory per hour Wow! So that means a 16GB laptop will on average experience 8 memory errors per hour under moderate load? I had no idea it was that high. This likely explains why my Mac crashes randomly but incredibly rarely. It's like once every few months. I guess a bit flip happens at the wrong place about that often?
  - taneliv 9y ago
    I would say that bit flipped in much less wrong place than the one that ended up corrupting some file on disk, unless you have a lot of insignificant data stored.
  - vbezhenar 9y ago
    That was a research long time ago. I don't think it's that bad now. But it's certainly a real threat.
deleted 9y ago
[deleted]
deleted 9y ago
[deleted]

sireat 9y ago

I believe there was data that with 64GB of RAM you can expect at least a single bit error or two a month. Thus ECC would be super helpful to those that expect that most of the 64GB will be filled with bits that shouldn't be changed. This article by a system integrator is a pretty decent data point: https://www.pugetsystems.com/labs/articles/Advantages-of-ECC-Memory-520/ https://www.pugetsystems.com/labs/articles/Advantages-of-ECC...

smitherfield 9y ago
>>>I believe there was data that with 64GB of RAM you can expect at least a single bit error or two a month. But this would be if you were using 64GB nonstop for a month, which is quite a bit rarer than just having 64GB. And certain filesystems (ZFS, Btrfs) and most databases have software checksumming functionality to detect bitrot. Not to say ECC isn't useful or multiple layers of protection aren't prudent, just that the problem it solves is something you're unlikely to encounter below datacenter scale.

ljcn 9y ago

Some research yes. To answer your final question directly - more like hourly. Google did a big study in 2009 [0] and found error rates between 2.4E-11 and 6.7E-11 errors/bit-hour, or, in more familiar units: about 1 error per 2-GByte of memory per hour. Obviously most errors go unnoticed, but for anything where random bit flips are unacceptable, ECC is necessary. Note too that error rates go up with memory density, not down, so things might be worse with more recent chips. [0] http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf http://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf

api 9y ago
> about 1 error per 2-GByte of memory per hour Wow! So that means a 16GB laptop will on average experience 8 memory errors per hour under moderate load? I had no idea it was that high. This likely explains why my Mac crashes randomly but incredibly rarely. It's like once every few months. I guess a bit flip happens at the wrong place about that often?
- taneliv 9y ago
  I would say that bit flipped in much less wrong place than the one that ended up corrupting some file on disk, unless you have a lot of insignificant data stored.
- vbezhenar 9y ago
  That was a research long time ago. I don't think it's that bad now. But it's certainly a real threat.

deleted 9y ago

[deleted]

deleted 9y ago

[deleted]