5 ms·

I don't think Threadripper is a hard requirement for ECC. There's some pretty reasonable TDP processors if you step down from Threadripper.

cuu508 6y ago
I haven't seen definite details and test results on these (but haven't looked recently). What specific configurations (CPU, MB, RAM) are known to work? Let's say I have a Ryzen system, how can I check if ECC really works? Like, can I see how many bit flips got corrected in, say, last 24h?
- xxs 6y ago
  Every Ryzen (non APU) supports it* Check the montherboard of your choice, they would declare it in big bold letters, e.g.[0] *not officially, and the memory controller provides no report for 'fixed' errors. 0: http://www.asrock.com/mb/AMD/X570%20Taichi/ http://www.asrock.com/mb/AMD/X570%20Taichi/
- cturner 6y ago
  Regarding verification. There is a debian package called edac-utils. As I recall you overclock your RAM and run your system at load in order to generate failures. Looking back at my notes, the output of journalctl -b tells should say something like, "Node 0: DRAM ECC enabled." Then 'edac-ctl --status' should tell you that drivers are loaded. Then you run 'edac-util -v' to report on what it has seen, mc0: 0 Uncorrected Errors with no DIMM info mc0: 0 Corrected Errors with no DIMM info mc0: csrow2: 0 Uncorrected Errors mc0: csrow2: mc#0csrow#2channel#0: 0 Corrected Errors mc0: csrow2: mc#0csrow#2channel#1: 0 Corrected Errors mc0: csrow3: 0 Uncorrected Errors mc0: csrow3: mc#0csrow#3channel#0: 0 Corrected Errors mc0: csrow3: mc#0csrow#3channel#1: 0 Corrected Errors edac-util: No errors to report.
  - a1369209993 6y ago
    > As I recall you overclock your RAM and run your system at load in order to generate failures. You can also use memtest86+ for this, although I don't recall if it requires specific configuration for ECC testing.
- p_l 6y ago
  All AMD CPUs with integrated memory controllers support ECC. The CPU also exposes an interface usable by the operating system to verify ECC works - the same interface is used to provide monitoring of memory fault data provided by ECC. They aren't tested on it, so it's possible to get a dud, but it's minuscule chance that isn't worth bothering. Now, to actual issues you can encounter: motherboards The problem is that ECC means you need to have, iirc, 8 more data lines between CPU and memory module, which of course mean more physical connections (don't remember how many right now). Those also need to be properly done and tested, and you might encounter a motherboard where it wasn't done. Not sure how common, unfortunately. Another issue is motherboard firmware. Even though AMD supplies the memory init code, the configuration can be tweaked by motherboard vendor, and they might simply break ECC support accidentally (even by something as simple as making a toggle default to false then forgot to expose it in configuration menu). Those are the two issues you can encounter. The difference with AFAIK Threadripper PRO, and EPYC, is that AMD includes ECC in its test and certification programs for it, which kind of enforces support.
  - jtl999 6y ago
    > Another issue is motherboard firmware. Even though AMD supplies the memory init code, the configuration can be tweaked by motherboard vendor, and they might simply break ECC support accidentally (even by something as simple as making a toggle default to false then forgot to expose it in configuration menu). I think some Gigabyte boards are infamous for this in certain circle OTOH: Gigabyte might have a Threadripper PRO motherboard (WRX80 chipset) coming out in the future
    - p_l 6y ago
      Gigabyte is also infamous for trying to claim that they implemented UEFI by dropping a build of DUET (UEFI that boots on top of BIOS, used for early development) into BIOS image...
- theevilsharpie 6y ago
  On Windows, to check if ECC is working, run the command 'wmic memphysical get memoryerrorcorrection': PC C:\> wmic memphysical get memoryerrorcorrection MemoryErrorCorrection 6 SuperUser has a convenient decoder[1], but modern systems will report "6" here if ECC is working. When Windows detects a memory error, it will record it in the system event log, under the WHEA source. As a side note, this is also how memory errors within the CPU's caches are reported under Windows. [1] https://superuser.com/questions/893560/how-do-i-tell-if-my-memory-is-ecc-or-non-ecc/893563#893563 https://superuser.com/questions/893560/how-do-i-tell-if-my-m...
- adrian_b 6y ago
  All desktop Ryzen CPUs without integrated GPU, i.e. with the exception of APUs, support ECC. You must check the specifications of the motherboard to see if ECC memory is supported. As a rule, all ASRock MBs support ECC and also some ASUS MBs support ECC, e.g. all ASUS workstation motherboards. I have no experience with Windows and Ryzen, but I assume that ECC should work also there. With Linux, you must use a kernel with all the relevant EDAC options enabled, including CONFIG_EDAC_AMD64. For the new Zen 3 CPUs, i.e. Ryzen 5xxx, you must use a kernel 5.10 or later, for ECC support. On Linux, there are various programs, e.g. edac-utils, to monitor the ECC errors. To be more certain that the ECC error reporting really works, the easiest way is to change the BIOS settings to overclock the memory, until memory errors appear.
usefulcat 6y ago
It’s not. I have a low end Epyc machine with ECC. It has a TDP of something like 30 watts.
- defanor 6y ago
  I didn't consider embedded CPUs (I guess that's about an embedded EPYC, not a server one), those look neat. But there's no official ECC support (i.e., it's similar to Ryzen CPUs), is there? Edit: as detaro mentioned in the reply, there is, and here's the source [0] -- that's what they mean by "RAS" on promotional pages [1]. That indeed looks like a nice option. [0] https://www.amd.com/system/files/documents/updated-3000-family-product-brief.pdf https://www.amd.com/system/files/documents/updated-3000-fami... [1] https://www.amd.com/en/products/embedded-epyc-3000-series https://www.amd.com/en/products/embedded-epyc-3000-series
  - loeg 6y ago
    RAS covers more than just DRAM, but yes. Historically, the reporting interface is called MCA (Machine Check Architecture) / MCE. I think both AMD and Intel have extensions with other names, but MCA/MCE points you in the right direction.
  - detaro 6y ago
    All EPYC, including the embedded ones, do officially have ECC support
  - adrian_b 6y ago
    For embedded applications, there is official ECC support for all CPUs named Epyc or Ryzen Vxxxx or Ryzen Rxxxx. There are computers in the Intel NUC form factor, with ECC support (e.g. with Ryzen V2718), e.g from ASRock Industrial.
- detaro 6y ago
  what kind of machine is that? Been vaguely looking for one a while back, and everything seemed difficult to get (since the main target is large-volume customers I guess)

cuu508 6y ago

I haven't seen definite details and test results on these (but haven't looked recently). What specific configurations (CPU, MB, RAM) are known to work? Let's say I have a Ryzen system, how can I check if ECC really works? Like, can I see how many bit flips got corrected in, say, last 24h?

xxs 6y ago
Every Ryzen (non APU) supports it* Check the montherboard of your choice, they would declare it in big bold letters, e.g.[0] *not officially, and the memory controller provides no report for 'fixed' errors. 0: http://www.asrock.com/mb/AMD/X570%20Taichi/ http://www.asrock.com/mb/AMD/X570%20Taichi/
cturner 6y ago
Regarding verification. There is a debian package called edac-utils. As I recall you overclock your RAM and run your system at load in order to generate failures. Looking back at my notes, the output of journalctl -b tells should say something like, "Node 0: DRAM ECC enabled." Then 'edac-ctl --status' should tell you that drivers are loaded. Then you run 'edac-util -v' to report on what it has seen, mc0: 0 Uncorrected Errors with no DIMM info mc0: 0 Corrected Errors with no DIMM info mc0: csrow2: 0 Uncorrected Errors mc0: csrow2: mc#0csrow#2channel#0: 0 Corrected Errors mc0: csrow2: mc#0csrow#2channel#1: 0 Corrected Errors mc0: csrow3: 0 Uncorrected Errors mc0: csrow3: mc#0csrow#3channel#0: 0 Corrected Errors mc0: csrow3: mc#0csrow#3channel#1: 0 Corrected Errors edac-util: No errors to report.
- a1369209993 6y ago
  > As I recall you overclock your RAM and run your system at load in order to generate failures. You can also use memtest86+ for this, although I don't recall if it requires specific configuration for ECC testing.
p_l 6y ago
All AMD CPUs with integrated memory controllers support ECC. The CPU also exposes an interface usable by the operating system to verify ECC works - the same interface is used to provide monitoring of memory fault data provided by ECC. They aren't tested on it, so it's possible to get a dud, but it's minuscule chance that isn't worth bothering. Now, to actual issues you can encounter: motherboards The problem is that ECC means you need to have, iirc, 8 more data lines between CPU and memory module, which of course mean more physical connections (don't remember how many right now). Those also need to be properly done and tested, and you might encounter a motherboard where it wasn't done. Not sure how common, unfortunately. Another issue is motherboard firmware. Even though AMD supplies the memory init code, the configuration can be tweaked by motherboard vendor, and they might simply break ECC support accidentally (even by something as simple as making a toggle default to false then forgot to expose it in configuration menu). Those are the two issues you can encounter. The difference with AFAIK Threadripper PRO, and EPYC, is that AMD includes ECC in its test and certification programs for it, which kind of enforces support.
- jtl999 6y ago
  > Another issue is motherboard firmware. Even though AMD supplies the memory init code, the configuration can be tweaked by motherboard vendor, and they might simply break ECC support accidentally (even by something as simple as making a toggle default to false then forgot to expose it in configuration menu). I think some Gigabyte boards are infamous for this in certain circle OTOH: Gigabyte might have a Threadripper PRO motherboard (WRX80 chipset) coming out in the future
  - p_l 6y ago
    Gigabyte is also infamous for trying to claim that they implemented UEFI by dropping a build of DUET (UEFI that boots on top of BIOS, used for early development) into BIOS image...
theevilsharpie 6y ago
On Windows, to check if ECC is working, run the command 'wmic memphysical get memoryerrorcorrection': PC C:\> wmic memphysical get memoryerrorcorrection MemoryErrorCorrection 6 SuperUser has a convenient decoder[1], but modern systems will report "6" here if ECC is working. When Windows detects a memory error, it will record it in the system event log, under the WHEA source. As a side note, this is also how memory errors within the CPU's caches are reported under Windows. [1] https://superuser.com/questions/893560/how-do-i-tell-if-my-memory-is-ecc-or-non-ecc/893563#893563 https://superuser.com/questions/893560/how-do-i-tell-if-my-m...
adrian_b 6y ago
All desktop Ryzen CPUs without integrated GPU, i.e. with the exception of APUs, support ECC. You must check the specifications of the motherboard to see if ECC memory is supported. As a rule, all ASRock MBs support ECC and also some ASUS MBs support ECC, e.g. all ASUS workstation motherboards. I have no experience with Windows and Ryzen, but I assume that ECC should work also there. With Linux, you must use a kernel with all the relevant EDAC options enabled, including CONFIG_EDAC_AMD64. For the new Zen 3 CPUs, i.e. Ryzen 5xxx, you must use a kernel 5.10 or later, for ECC support. On Linux, there are various programs, e.g. edac-utils, to monitor the ECC errors. To be more certain that the ECC error reporting really works, the easiest way is to change the BIOS settings to overclock the memory, until memory errors appear.

usefulcat 6y ago

It’s not. I have a low end Epyc machine with ECC. It has a TDP of something like 30 watts.

defanor 6y ago
I didn't consider embedded CPUs (I guess that's about an embedded EPYC, not a server one), those look neat. But there's no official ECC support (i.e., it's similar to Ryzen CPUs), is there? Edit: as detaro mentioned in the reply, there is, and here's the source [0] -- that's what they mean by "RAS" on promotional pages [1]. That indeed looks like a nice option. [0] https://www.amd.com/system/files/documents/updated-3000-family-product-brief.pdf https://www.amd.com/system/files/documents/updated-3000-fami... [1] https://www.amd.com/en/products/embedded-epyc-3000-series https://www.amd.com/en/products/embedded-epyc-3000-series
- loeg 6y ago
  RAS covers more than just DRAM, but yes. Historically, the reporting interface is called MCA (Machine Check Architecture) / MCE. I think both AMD and Intel have extensions with other names, but MCA/MCE points you in the right direction.
- detaro 6y ago
  All EPYC, including the embedded ones, do officially have ECC support
- adrian_b 6y ago
  For embedded applications, there is official ECC support for all CPUs named Epyc or Ryzen Vxxxx or Ryzen Rxxxx. There are computers in the Intel NUC form factor, with ECC support (e.g. with Ryzen V2718), e.g from ASRock Industrial.
detaro 6y ago
what kind of machine is that? Been vaguely looking for one a while back, and everything seemed difficult to get (since the main target is large-volume customers I guess)