4 ms·

The performance penalties I'm referring to are memory performance penalties. If your workload isn't memory-bound (and most aren't), you'll measure a difference

The performance penalties I'm referring to are memory performance penalties. If your workload isn't memory-bound (and most aren't), you'll measure a difference of 2% or less, sure, but that's not really relevant when talking about memory performance. In extreme cases (such as, for instance, lattice Boltzmann simulations, where the streaming step is completely memory bound) I've seen over 50% performance degradation, but much of it can be attributed to the notoriously loose timings on most ECC DRAM.

Retric 9y ago
[Citation Needed] With identical memory ECC and non <2% is the actual memory overhead. Now, non ECC memory might have better specs / a higher overclock rate etc, but doing either of those increases the rate of errors. And sure you might not notice the problem directly, but that does not mean things are ok.
dijit 9y ago
I feel I have the relevant real world expertise to have a say about that. I am one of the principle engineers working on a game that was able to support over 1.2 million players concurrently, so I feel that I have some merit behind what I'm about to say. ECC Memory does not have such an extreme overhead. Our gameserver is memory bandwidth bound. To put that into perspective a change from DDR3 to DDR4 saw a jump in performance of 34%~. Using non-ecc memory (which is obviously very available to the development team) only gave a penalty of roughly 3% in the worst case. (both DDR4 systems). Why are we memory bandwidth bound? because we allocate 250GB of "game world" and phases and make teeny tiny modifications to them when you run around or pick up loot and shit. It's as taxing as you can be on memory bandwidth since you can't batch set or load anything.
- eightysixfour 9y ago
  I think what he is trying to say is there isn’t a lot of DDR4 out there at 3200mhz with tight timings like there is with non-ECC memory, not just that ECC vs non-ECC at the same specs is 10-30% slower.
  - keldaris 9y ago
    That's right, I'm sorry if I worded it ambiguously in my previous posts. ECC RAM is considerably slower than non-ECC RAM, but nowadays it's mostly because of looser timings and lower frequencies. I'm well aware that there are people who claim that frequencies beyond the ECC standard are liable to cause data loss (for reasons usually left unspecified), but I've never seen any reliable data to support that claim, as long as the frequencies and timings tested by the manufacturer are maintained (not to be confused with people who overclock RAM way beyond its tested specification and experience instability). On a further note, while the implementation of ECC itself is very fast for modern RAM, the same is not true of VRAM on GPUs. At least as implemented in NVidia Tesla GPUs, enabling ECC has a huge performance penalty by itself. [1] [1] http://geoceano.fr/wp-content/uploads/2015/01/cpu-gpu-lbm-in-heterogeneous-clusters.pdf http://geoceano.fr/wp-content/uploads/2015/01/cpu-gpu-lbm-in...
    - Retric 9y ago
      I think some of this breakdown is from motherboard support. You can MB that supports a single CPU and 128GB for DDR4 4400 which is significantly higher than ECC. When you start looking at multi CPU computers with 1+TB of ram the story changes. PS: Some of this stuff is also really hard to test. Without ECC it's hard to data in terms of stability for petabytes of ram over years. And no extrapolations don't work very well.

Retric 9y ago

[Citation Needed] With identical memory ECC and non <2% is the actual memory overhead. Now, non ECC memory might have better specs / a higher overclock rate etc, but doing either of those increases the rate of errors. And sure you might not notice the problem directly, but that does not mean things are ok.

dijit 9y ago

I feel I have the relevant real world expertise to have a say about that. I am one of the principle engineers working on a game that was able to support over 1.2 million players concurrently, so I feel that I have some merit behind what I'm about to say. ECC Memory does not have such an extreme overhead. Our gameserver is memory bandwidth bound. To put that into perspective a change from DDR3 to DDR4 saw a jump in performance of 34%~. Using non-ecc memory (which is obviously very available to the development team) only gave a penalty of roughly 3% in the worst case. (both DDR4 systems). Why are we memory bandwidth bound? because we allocate 250GB of "game world" and phases and make teeny tiny modifications to them when you run around or pick up loot and shit. It's as taxing as you can be on memory bandwidth since you can't batch set or load anything.

eightysixfour 9y ago
I think what he is trying to say is there isn’t a lot of DDR4 out there at 3200mhz with tight timings like there is with non-ECC memory, not just that ECC vs non-ECC at the same specs is 10-30% slower.
- keldaris 9y ago
  That's right, I'm sorry if I worded it ambiguously in my previous posts. ECC RAM is considerably slower than non-ECC RAM, but nowadays it's mostly because of looser timings and lower frequencies. I'm well aware that there are people who claim that frequencies beyond the ECC standard are liable to cause data loss (for reasons usually left unspecified), but I've never seen any reliable data to support that claim, as long as the frequencies and timings tested by the manufacturer are maintained (not to be confused with people who overclock RAM way beyond its tested specification and experience instability). On a further note, while the implementation of ECC itself is very fast for modern RAM, the same is not true of VRAM on GPUs. At least as implemented in NVidia Tesla GPUs, enabling ECC has a huge performance penalty by itself. [1] [1] http://geoceano.fr/wp-content/uploads/2015/01/cpu-gpu-lbm-in-heterogeneous-clusters.pdf http://geoceano.fr/wp-content/uploads/2015/01/cpu-gpu-lbm-in...
  - Retric 9y ago
    I think some of this breakdown is from motherboard support. You can MB that supports a single CPU and 128GB for DDR4 4400 which is significantly higher than ECC. When you start looking at multi CPU computers with 1+TB of ram the story changes. PS: Some of this stuff is also really hard to test. Without ECC it's hard to data in terms of stability for petabytes of ram over years. And no extrapolations don't work very well.