5 ms·
ECC adds significant cost, and the benefits are stastically meager. I ran a fleet of a few thousand servers with all having 64GB and some having up to 768GB ECC
by toast0 6y ago
ECC adds significant cost, and the benefits are stastically meager. I ran a fleet of a few thousand servers with all having 64GB and some having up to 768GB ECC RAM. I'd estimate we had no more than one swap a month. Most systems never had any (reported) ECC corrections.
I still think it's important, but it makes sense that people don't want to pay more for something to detect errors that are quite unlikely.
- dmm 6y agoWhat do you mean by ECC is more expensive? Are you talking about ECC ram being more expensive? That's probably true but why not give people the option whether to use ECC? ECC being optional is a good idea anyway because ECC memory is generally slower. I don't think the actual ECC implementation in the processor is very expensive to implement. Consumer CPUs not having ECC is pure product segmentation, without technical justification, IMO.
- toast0 6y agoECC ram is more expensive, yes. But supporting ECC ram also adds more expense. To properly support it, BIOS engineers have to support and test it; motherboard makers have to support and test it. I appreciate that AMD offers it for their consumer oriented processors, but because it's a best effort feature, and it's hard for an end user (or reviewer) to test, you never really know if you're going to get full support, or if the support is really just that you can use ECC ram, if you want to spend a little more for your ram, and get none of the benefits of ECC. It certainly adds something to the cost (die space) of the memory controller, but I agree it's probably not much.
- cesaref 6y agoWell those costs are mainly related to adding support for another memory standard, but i'm suggesting ECC become the standard with non-ECC being dropped. The BIOS/testing cost would be unaffected, they would just be testing only ECC memory. Realistically the mainstream CPU manufacturers have ECC solutions, so including them in the mainstream processors shouldn't be a huge issue, it's just a market segmentation ploy to exclude the feature from the consumer processor designs.
- toast0 6y ago> i'm suggesting ECC become the standard with non-ECC being dropped. I think the cost pressure is too high. Early IBM PCs required parity ram (a 9th chip to store if the sum of bits was even or odd), and would fault if the value was incorrect on reads. Ram module manufactures made innovative fake parity modules that calculated the parity value on access, replacing the 9th ram chip with a very simple circuit and saving money. It would be hard to convince the whole industry not to make fake ECC ram, if ECC was mandatory.
- StillBored 6y agoFake ECC as you put it serves a purpose, it protects the bus interface. Which is one of the failure points on modern machines, and is why sometimes to fix ram/qpi/etc errors you end up replacing the motherboard. We will have to see with DDR5 (because it supports "internal" ECC) if its worth it to the memory industry to build RAM that is internally denser, but more error prone (as is the case with modern flash) or continue to attempt to build 100% reliable ram (and failing). I'm betting some clever person figures that out. Which leaves only the memory bus itself unprotected. Which IMHO, is foolish and serves only to create product segmentation. So, for a DDR5 dimm with internal ECC, generating bus ECC should be a trivial addition.
- a1369209993 6y ago> that calculated the parity value on access, replacing the 9th ram chip with a very simple circuit and saving money. > It would be hard to convince the whole industry not to make fake ECC ram, if ECC was mandatory. Assuming you don't use memory-mapped IO, that's easy to fix. On startup, generate 4 random bits a,b,c,d. Parity bit is data line 4a+2b+c, with d?even:odd parity, data bit 4a+2b+c is on data line D8. ECC on 64/72 uses more random bits, but is otherwise similar, although for modern chipsets it would probably have to be scrambled in the northbridge or southbridge (or equivalent) rather than the CPU, to allow for DMA and such. Note that there's no gate delay involved here; the multiplexing can be done with pass transistors.
- jrockway 6y agoWell, it uses 12.5% of the bytes for error correction instead of data storage, which is not very much overhead. Your 128GB becomes 112GB, which is still pretty good. I also disagree that the benefits are meager. I have suffered from several computers with flaky memory (fine when you build the computer, flaky several years later; then you do an overnight memory test and find that indeed the memory has gone bad), and a strong software signal that "hey your memory is bad" is very actionable. You also have to think about it from a programming standpoint -- what happens if this variable isn't what I set it to? What if "for 1..10" is actually "for 1..2147483658"? Do you have time to debug that? How much data do you lose when you persist that to disk? To me it is insane to not get this nearly-free consistency check if you ever plan to persist any bytes in RAM to long-term storage. Even consumer GPUs have ECC memory these days. It's a no-brainer. What surprises me is that the industry segments RAM sticks by ECC/not-ECC, when it's just a software function performed by the memory controller. I think everyone with a HEDT setup would be happy to enable ECC and get better reliability at the cost of 12% less RAM. I know I would be. (I built a Threadripper workstation recently and just couldn't get any reasonable prices on QVL'd ECC memory. So I skipped it and paid like $600 for 128GB of 3600MT/16-CAS memory... would be happy to flip a switch and have that be only 112GB.) The reason why ECC isn't widespread is it's because it's used to segment the market. Home users are cost sensitive, so hardware vendors have to think of ways to get server users to not buy home user equipment. ECC is one of these levers. That's all there is to it. Everyone would have ECC if it were a mere 12.5% more expensive.
- Thetawaves 6y agoThis is not how ECC ram works. 128GB of ram does not change to 112GB with ECC. 128GB of ram with ECC will invariably be 144GB. Standard memory modules become (a multiple of) 9 bits wide, with the additional bit stored near the other 8 bits.
- StillBored 6y agoWhich is still 12.5%. Which for most people wouldn't be a problem since ram is a fraction of the system price (so it might only raise the price of the machine 3% if the ram is 1/3 of the total BOM). The problem is _INTEL_ deciding its a premium feature, and the memory manufactures charging 50%+ more for 12.5% more hardware. So instead of being a couple percent of the system cost, it ends up being a noticeable double digit percentage. Of course none of this explains why apple/etc haven't done it in their phones.
- bcrl 6y agoECC is only more expensive until the first time you have to spend multiple hours debugging an issue or recovering data due to memory corruption. My time and data are worth more than the $40 premium paid for the ECC unbuffered DIMMs in my Ryzen system. And that price premium would pretty much go away if every system shipped with ECC memory as the DRAM vendors would produce x9 chips instead of x8. Shame on Intel for segmenting ECC memory out of the consumer market.
- glandium 6y ago> ECC adds significant cost Does it, though? I don't remember all the prices from when I was researching memory close to a year ago for a new PC, but one big notable difference between ECC and non-ECC I've seen is that almost all non-ECC sticks are overclocked. You buy something rated at 2667MHz, and you actually get is e.g. 1833MHz chips overclocked at that speed. Which makes them cheaper. But I'd expect non-overclocked non-ECC sticks to be that far in pricing to ECC sticks (well, apart from the obvious difference they'd have more chips). And because the market is focused on cheap non-ECC sticks, they don't do cheaper ECC overclocked sticks, but there is no reason they couldn't make them. There aren't that many ECC unbuffered sticks already, it's easier to find registered ones. Essentially, the market is skewed by Intel not supporting ECC on non-Xeons.
- account42 6y agoIronically, ECC is great to have for manual overclocking as you will know when you pushed your system too far - even if most things work fine and problems only happen with some demanding workloads. Without ECC you'd likely falsely assume that the software / OS / drivers are buggy since everything else works.
- CrLf 6y agoBenefits are statistically meager until a bit silently flips somewhere important and you only discover it days later when it has propagated into massive data corruption. Been there. If your servers don’t have ECC memory, you’re eventually going to get bit.