5 ms·

> so long as the big players see it as a "Professional User" product differentiation point it'll always be more expensive than it should be. Right. The more im

> so long as the big players see it as a "Professional User" product differentiation point it'll always be more expensive than it should be. Right. The more important Linus to speak up for ECC isn't Torvalds. It's Linus Sebastian, of Linus Tech Tips. He's made a few videos on ECC targeted towards gamers. Gamers drive the enthusiast PC market and when they start caring, more ECC gets made which will drive the cost down a bit. Last time I bought 32GB DDR4 UDIMM ECC there was literally one SKU. Not manufacturer. Not brand. SKU. One single item in production in the entire world. 16GB wasn't much better off, either. It's a hard sell, though. Non-ECC will always be cheaper because it costs less to produce. Gamers don't really care that ECC prevents one crash in years because they are used to frequent crashes already. They are largely being fed dogshit from the AAA gaming industry and they have learned to just deal with it. Crashes are just part of being on the bleeding edge of gaming and Nvidia/Radeon drivers. One less crash in a sea of crashes isn't something gamers are lining up for. But a better model GPU or bigger SSD? It's an obvious choice.

kimixa 5y ago
> Gamers don't really care that ECC prevents one crash in years because they are used to frequent crashes already. I work on GPU drivers for one of those companies. We regularly get reports and backtraces that cannot be reproduced, or "Cannot Happen" without some external factor (e.g. some other bit of code poking around our memory space). Often they're just silently dropped or ignored on the long tail of issues that nobody can get any traction on. My understanding is the stats from hyperscalers is that ECC correction events happen a lot more than "Common Knowledge" may imply - I wonder just what proportion of things that are blamed on software may actually be due to hardware issues like this? Again, without a significant change in the market (IE enough gamers start using ECC to actually be statistically relevant and comparing stability) this cannot really be tested, but I've wondered.
- bcrl 5y ago
  Except that anyone using Intel desktop CPUs pretty much can't use ECC thanks to marketing deciding that ECC is a market segmentation feature. The real way to make ECC happen industry wide is for OS vendors like Microsoft to make it a platform requirement. A no ECC, no boot policy would change things overnight. Sadly, we can't even get DRAM manufacturers to fix row hammer properly, so the likelihood of this happening is pretty much nil.
  - wmf 5y ago
    You can use ECC by buying the Xeon version which is only slightly more expensive.
    - temac 5y ago
      Though the motherboards and ram modules are also harder to find and with far less choice. (and actually the processors are too, depending on your market)
      - sliken 5y ago
        Yes, sure, motherboards and CPUs have far less choices. But generally plenty for any normal use case. Motherboards and CPUs have a crazy number of flavors, none of which matter one whit for normal use cases. It's so bad motherboards are differentiating on the manliness of the heat sink design, lighting options, and the color of the motherboard.
  - sliken 5y ago
    If people cared, they would buy ECC capable chips. In fact my desktop is a Xeon e3-1230v5, which as cheaper and slightly slower (3.4 vs 3.6 GHz or something) then the equivalent i7. It was $50 more for the motherboard and $100 more for the ram. I'm sure if the market flooded to ECC capable chips (the silicon is the same) Intel would sell them. So many people grumble, but I'm not really sure Intel should push ECC if desktops users aren't willing to pay a modest premium for it. Many cheer AMD, which does not disable ECC on desktop chips, but neither do they promise ECC will actually work. It's a confusing mess between physical capacity (ram increases by 16GB when you add a 16GB dimm), and the actual correction of errors and telling the OS about the event. Only on the EPYC does AMD test and certify that ECC will work.
    - bcrl 5y ago
      Your comments about AMD are not entirely true: AMD officially supports ECC in the Pro variants of their desktop CPUs / APUs. I'm currently using a Ryzen 3800X with ECC memory and EDAC reports that it's working. ASRock lists qualified ECC DIMMs for virtually all of their motherboards. It's more that it's a feature motherboard vendors can support if they want to rather than being a platform requirement. But yes, it is sad that AMD doesn't make it an officially supported feature of the platform.
- temac 5y ago
  > My understanding is the stats from hyperscalers is that ECC correction events happen a lot more than "Common Knowledge" may imply - I wonder just what proportion of things that are blamed on software may actually be due to hardware issues like this? What is the order of magnitude of ECC correction events?
  - wmf 5y ago
    https://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf https://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf
    - sliken 5y ago
      Looks like a good paper, but transistors have gotten quite a bit smaller since 2009, and voltages have gotten lower as well. So the energy required to flip a bit has shrunk quite a bit since 2009.
kmeisthax 5y ago
Linus[0] already made a video agreeing with Linus[1]'s rant: https://www.youtube.com/watch?v=pPeCNrNTr3k https://www.youtube.com/watch?v=pPeCNrNTr3k I can actually give you a GAMING (or at least enthusiast) use case for ECC, too: memory overclock validation. Right now, it's kind of a dark art, but ECC failures would absolutely give you a very explicit early warning that you're pushing the CPU IMC too hard. [0] Pronounced LIE-nus [1] Pronounced lee-nus
- kimixa 5y ago
  I can also tell you in a 'gamer'-adjacent field a very large proportion of users have a very loose definition of stability when overclocking. It survived a single short run of a single use case that only runs one load pattern? Must be perfectly stable. Anything else crashes? It must be the shitty game or gpu driver! I hope that ecc will give more warning that you're close to the edge (e.g. any ecc correction means it's unstable and you should back off), but fear people will just keep pushing until the ecc can no longer even correct the errors. Professionally I work on gpu drivers for systems that do not allow overclocking, so shouldn't affect our stats, but it certainly is a mess for other teams. I've noticed that it's even become a common suggestion on fan forums for gpu manufacturers to disable any overclock and revalidate when changing vendors or even updating drivers, presumably as they're so close to the edge small changes in the access patterns will then cause it to explode.

kimixa 5y ago

> Gamers don't really care that ECC prevents one crash in years because they are used to frequent crashes already. I work on GPU drivers for one of those companies. We regularly get reports and backtraces that cannot be reproduced, or "Cannot Happen" without some external factor (e.g. some other bit of code poking around our memory space). Often they're just silently dropped or ignored on the long tail of issues that nobody can get any traction on. My understanding is the stats from hyperscalers is that ECC correction events happen a lot more than "Common Knowledge" may imply - I wonder just what proportion of things that are blamed on software may actually be due to hardware issues like this? Again, without a significant change in the market (IE enough gamers start using ECC to actually be statistically relevant and comparing stability) this cannot really be tested, but I've wondered.

bcrl 5y ago
Except that anyone using Intel desktop CPUs pretty much can't use ECC thanks to marketing deciding that ECC is a market segmentation feature. The real way to make ECC happen industry wide is for OS vendors like Microsoft to make it a platform requirement. A no ECC, no boot policy would change things overnight. Sadly, we can't even get DRAM manufacturers to fix row hammer properly, so the likelihood of this happening is pretty much nil.
- wmf 5y ago
  You can use ECC by buying the Xeon version which is only slightly more expensive.
  - temac 5y ago
    Though the motherboards and ram modules are also harder to find and with far less choice. (and actually the processors are too, depending on your market)
    - sliken 5y ago
      Yes, sure, motherboards and CPUs have far less choices. But generally plenty for any normal use case. Motherboards and CPUs have a crazy number of flavors, none of which matter one whit for normal use cases. It's so bad motherboards are differentiating on the manliness of the heat sink design, lighting options, and the color of the motherboard.
- sliken 5y ago
  If people cared, they would buy ECC capable chips. In fact my desktop is a Xeon e3-1230v5, which as cheaper and slightly slower (3.4 vs 3.6 GHz or something) then the equivalent i7. It was $50 more for the motherboard and $100 more for the ram. I'm sure if the market flooded to ECC capable chips (the silicon is the same) Intel would sell them. So many people grumble, but I'm not really sure Intel should push ECC if desktops users aren't willing to pay a modest premium for it. Many cheer AMD, which does not disable ECC on desktop chips, but neither do they promise ECC will actually work. It's a confusing mess between physical capacity (ram increases by 16GB when you add a 16GB dimm), and the actual correction of errors and telling the OS about the event. Only on the EPYC does AMD test and certify that ECC will work.
  - bcrl 5y ago
    Your comments about AMD are not entirely true: AMD officially supports ECC in the Pro variants of their desktop CPUs / APUs. I'm currently using a Ryzen 3800X with ECC memory and EDAC reports that it's working. ASRock lists qualified ECC DIMMs for virtually all of their motherboards. It's more that it's a feature motherboard vendors can support if they want to rather than being a platform requirement. But yes, it is sad that AMD doesn't make it an officially supported feature of the platform.
temac 5y ago
> My understanding is the stats from hyperscalers is that ECC correction events happen a lot more than "Common Knowledge" may imply - I wonder just what proportion of things that are blamed on software may actually be due to hardware issues like this? What is the order of magnitude of ECC correction events?
- wmf 5y ago
  https://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf https://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf
  - sliken 5y ago
    Looks like a good paper, but transistors have gotten quite a bit smaller since 2009, and voltages have gotten lower as well. So the energy required to flip a bit has shrunk quite a bit since 2009.

kmeisthax 5y ago

Linus[0] already made a video agreeing with Linus[1]'s rant: https://www.youtube.com/watch?v=pPeCNrNTr3k https://www.youtube.com/watch?v=pPeCNrNTr3k I can actually give you a GAMING (or at least enthusiast) use case for ECC, too: memory overclock validation. Right now, it's kind of a dark art, but ECC failures would absolutely give you a very explicit early warning that you're pushing the CPU IMC too hard. [0] Pronounced LIE-nus [1] Pronounced lee-nus

kimixa 5y ago
I can also tell you in a 'gamer'-adjacent field a very large proportion of users have a very loose definition of stability when overclocking. It survived a single short run of a single use case that only runs one load pattern? Must be perfectly stable. Anything else crashes? It must be the shitty game or gpu driver! I hope that ecc will give more warning that you're close to the edge (e.g. any ecc correction means it's unstable and you should back off), but fear people will just keep pushing until the ecc can no longer even correct the errors. Professionally I work on gpu drivers for systems that do not allow overclocking, so shouldn't affect our stats, but it certainly is a mess for other teams. I've noticed that it's even become a common suggestion on fan forums for gpu manufacturers to disable any overclock and revalidate when changing vendors or even updating drivers, presumably as they're so close to the edge small changes in the access patterns will then cause it to explode.