5 ms·

I hope. It wouldn't be such a big issue if Intel stopped stripping ECC from desktop CPU's. Traditionally bad ram causes more bit rot than damaged drives. Ram ge

I hope. It wouldn't be such a big issue if Intel stopped stripping ECC from desktop CPU's. Traditionally bad ram causes more bit rot than damaged drives. Ram get less reliable every die shrink, everything should be ECC by now.

DaiPlusPlus 9y ago
CPU ECC is for RAM, not storage - checksums/parity is not the concern of the CPU at all. Though we're talking about Optane, then that's different. In the future (with solid OS support), Optane will be both RAM and non-volatile storage - and you really don't want data corruption there. Has Intel announced anything regarding ECC, parity or other systems in-place to protect against bitrot in Optane?
- shouldbworking 9y ago
  I don't think you get what I'm saying. The bit rot that happens to your data usually happens when it's in the RAM. The CPU likes it there when you're doing anything with it and once you're done editing a file it will stream happily corrupted data right back to disk. Hard drive and SSD error detection and correction is quite good, in regular ram its non existent. This is a major danger with database servers, as regular bit rot can be detected or corrected above the filesystem level, but no systems exist that can garantee protection to data in ram. Since databases try to do most/all operations in ram before flushing to disk there's a huge risk of silent data corruption. Adding bitrot protection to the filesystem only gives you more protection if the drive has crappy error correction. Adding protection to ram adds checks to a system that usually has none
  - gbrown_ 9y ago
    > The bit rot that happens to your data usually happens when it's in the RAM. Would love to read a source on this.
    - SamReidHughes 9y ago
      Disks have their own block checksumming so they can catch errors.
    - wtallis 9y ago
      Just check your disk's SMART indicators for unrecoverable errors. Drives are using pretty high quality checksums on data at rest; the chance of corruption silently slipping past both the BCH and LDPC codes used by a typical modern SSD is far smaller than the chance of your non-ECC RAM getting corrupted by cosmic rays.
      - PhantomGremlin 9y ago
        Your statement is correct, but doesn't tell the rest of the story. The disk has an onboard data buffer, usually implemented in DRAM. That DRAM chip is likely sourced by the lowest bidder, and it is likely not to have either parity or ECC. Also the CPU used in the drive can introduce its own data errors and corruption. It doesn't even have to be hardware. A firmware bug can also corrupt data. There are numerous buffers and storage elements involved in transferring the data to/from the disk's onboard memory to/from the computer's RAM. Many/most of those elements are not checked for parity. Data is subject to corruption in transit. That's what makes a filesystem like ZFS so interesting. It checks the entire data path. It doesn't care where the corruption occurs.
        shouldbworking 9y ago
        I never considered DRAM errors on the drive... Maybe that's one of the big cost differentiators of those super expensive commercial grade SSDs.
kalleboo 9y ago
I was under the impression that without ECC RAM, you should't be using an auto-repairing file system anyway because it will cause the bit rot to spread when it incorrectly calculates checksums.
- XorNot 9y ago
  This is incorrect. The point of ECC RAM is that if you are worried about bit-rot, you'll still get it without ECC RAM. Strictly speaking something like ZFS improves the situation - since a bit-flip which hits the checksum and not the data at the very least will raise an alert that something went wrong. But this will also happen with any other filesystem - it'll just be silent. But the point is no filesystem can protect you from unreliable RAM.
zamalek 9y ago
Why not just go with AMD? ECC is supported on all Ryzen SKUs.
- vetinari 9y ago
  But that does not mean that all the boards support it, even if the chip works at all. What good ECC RAM does, if the errors are not logged?
c3833174 9y ago
Sure probability of errors increases, but RAM reliability seems to have improved

DaiPlusPlus 9y ago

CPU ECC is for RAM, not storage - checksums/parity is not the concern of the CPU at all. Though we're talking about Optane, then that's different. In the future (with solid OS support), Optane will be both RAM and non-volatile storage - and you really don't want data corruption there. Has Intel announced anything regarding ECC, parity or other systems in-place to protect against bitrot in Optane?

shouldbworking 9y ago
I don't think you get what I'm saying. The bit rot that happens to your data usually happens when it's in the RAM. The CPU likes it there when you're doing anything with it and once you're done editing a file it will stream happily corrupted data right back to disk. Hard drive and SSD error detection and correction is quite good, in regular ram its non existent. This is a major danger with database servers, as regular bit rot can be detected or corrected above the filesystem level, but no systems exist that can garantee protection to data in ram. Since databases try to do most/all operations in ram before flushing to disk there's a huge risk of silent data corruption. Adding bitrot protection to the filesystem only gives you more protection if the drive has crappy error correction. Adding protection to ram adds checks to a system that usually has none
- gbrown_ 9y ago
  > The bit rot that happens to your data usually happens when it's in the RAM. Would love to read a source on this.
  - SamReidHughes 9y ago
    Disks have their own block checksumming so they can catch errors.
  - wtallis 9y ago
    Just check your disk's SMART indicators for unrecoverable errors. Drives are using pretty high quality checksums on data at rest; the chance of corruption silently slipping past both the BCH and LDPC codes used by a typical modern SSD is far smaller than the chance of your non-ECC RAM getting corrupted by cosmic rays.
    - PhantomGremlin 9y ago
      Your statement is correct, but doesn't tell the rest of the story. The disk has an onboard data buffer, usually implemented in DRAM. That DRAM chip is likely sourced by the lowest bidder, and it is likely not to have either parity or ECC. Also the CPU used in the drive can introduce its own data errors and corruption. It doesn't even have to be hardware. A firmware bug can also corrupt data. There are numerous buffers and storage elements involved in transferring the data to/from the disk's onboard memory to/from the computer's RAM. Many/most of those elements are not checked for parity. Data is subject to corruption in transit. That's what makes a filesystem like ZFS so interesting. It checks the entire data path. It doesn't care where the corruption occurs.
      - shouldbworking 9y ago
        I never considered DRAM errors on the drive... Maybe that's one of the big cost differentiators of those super expensive commercial grade SSDs.

kalleboo 9y ago

I was under the impression that without ECC RAM, you should't be using an auto-repairing file system anyway because it will cause the bit rot to spread when it incorrectly calculates checksums.

XorNot 9y ago
This is incorrect. The point of ECC RAM is that if you are worried about bit-rot, you'll still get it without ECC RAM. Strictly speaking something like ZFS improves the situation - since a bit-flip which hits the checksum and not the data at the very least will raise an alert that something went wrong. But this will also happen with any other filesystem - it'll just be silent. But the point is no filesystem can protect you from unreliable RAM.

zamalek 9y ago

Why not just go with AMD? ECC is supported on all Ryzen SKUs.

vetinari 9y ago
But that does not mean that all the boards support it, even if the chip works at all. What good ECC RAM does, if the errors are not logged?

c3833174 9y ago

Sure probability of errors increases, but RAM reliability seems to have improved