6 ms·

You still want ECC to make sure the data doesn't get corrupted on the way in or on the way out. ZFS will store what you send it, and it isn't responsible for yo

You still want ECC to make sure the data doesn't get corrupted on the way in or on the way out. ZFS will store what you send it, and it isn't responsible for your data once in hands it to you. The best part about ECC RAM, IMO, isn't the correction, but the checking. I want to know when my RAM goes bad, and I'd prefer to know before ZFS detects a problem.

blackflame7000 7y ago
I'm so sick of this argument every time a ZFS article comes up. This is by the inventor, Matthew Ahrens: “There’s nothing special about ZFS that requires/encourages the use of ECC RAM more so than any other filesystem.” https://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-your-data/ https://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-y... Now give me my karma back.
- dfeojm-zlib 7y ago
  The thing about ECC vs. non-ECC. If you don't have ECC, you don't know silent bit corruption is happening, and when it's finally (if it's ever) discovered, there's nothing that can be done. ECC (SECDED) keeps data safer than it would be otherwise. ECC is necessary for anything real or important based on the financial cost of losing data. For everything else, it can be optional. Plus, don't forget the security issues of bitsquatting and other attacks that are the real results of silent bitflips. DEF CON 19 - Artem Dinaburg - Bit-squatting: DNS Hijacking Without Exploitation https://youtu.be/9WcHsT97suU https://youtu.be/9WcHsT97suU
  - blackflame7000 7y ago
    All computers do error checking. Here is a summary: Non-ECC - Checksum: parity bit BlockSize: 1 byte ECC - Checksum: 7 bits BlockSize: 8 bytes ZFS - Checksum: 256 bits BlockSize: ~128kb
    - devonkim 7y ago
      It’s a lot further than this though with unbuffered RAM, registered RAM in the mix affecting performance and such. Error detection and error correction are different and require different levels of parity codes. Standard RAM at one point (SIMMs in the 90s) detected errors but won’t ever correct them. ECC can detect more but error patterns and now correct error patterns up to so many bits. The issues are more subtle beyond hardware though - your OS kernel has to understand the detection from the BIOS / EFI and if the motherboard manufacturer decided not to opt for wiring in the checksum fail signal you will run blind to data corruption that’s potentially really subtle on typical consumer hardware. With some BIOSes RAM failures result in a hard panic and will hard reboot the machine (the idea being a crash is better than continuing with an error). For me, ECC (both UDIMM and RDIMM) today are so inexpensive it’s a no-brainer for builds for business.
- tasty_freeze 7y ago
  The comment from the inventor isn't supporting your position. ZFS doens't need it any more, but if you care about data integrity, then you want ECC memory in your system. Yes, ZFS can take care of errors while it inside the ZFS API boundary, but the data on the outside of that is at risk, unless you have some higher level error correction protocol going on.
  - blackflame7000 7y ago
    Don't make the mistake of thinking Non-ECC ram doesn't have built-in error protection it's just not as robust. Random memory bit flips are pretty rare but it takes multiple at the same time for ECC to be effective. Additionally, the thing that makes no sense about ECC Ram is that you have no parity checks when data is in transit too and from the CPU(s). So a little corrosion on a trace or bad capacitor and ECC Ram won't be able to do anything about it.
    - agapon 7y ago
      It's you who is making the mistake. Non-ECC RAM does not have any protection. Prove me wrong, describe the mechanism.
      - blackflame7000 7y ago
        https://en.wikipedia.org/wiki/RAM_parity https://en.wikipedia.org/wiki/RAM_parity
        rincebrain 7y ago
        From that article: "The SDRAM and DDR modules that replaced the earlier types are usually available either without error-checking or with ECC (full correction, not just parity)."
        blackflame7000 7y ago
        "ECC (full correction, not just parity)" - meaning byte level parity is standard whereas a cache line checksum is extra.
- a1369209993 7y ago
  Can't speak for the downvoters, but I think what beatgammit is saying is that, in cases where you care about data corruption in ZFS, you also care about data corruption in parts of the system that aren't ZFS. Like the memory maps of every piece of application software. Nothing about ZFS needs ECC, but the circumstances ZFS claims to protect against need ECC.
  - blackflame7000 7y ago
    No ZFS never protects against data in transit. It makes 1 promise, that the bytes written to disk will be the same bytes read from disk at a later point in time. ZFS can't be responsible for being given bad data. Corrupt data looks just as writable as normal data to a storage controller. The article I linked shows how even bad ram isn't damaging to the integrity of a properly configured ZFS file system.
  - kissgyorgy 7y ago
    Exactly! You build a system with ZFS, because you need better defense against corruption, but without ECC, RAM will be the weak point in the system, so if you want the same level of guarantees, you need ECC, not because of ZFS.
    - blackflame7000 7y ago
      How so? You have a file from source A acquired via network, external storage, disc, etc and you want to store it on a ZFS array. If the ram corrupts the file before ZFS can store it, then the destination checksum will not match the source checksum. In reverse, you load the file from ZFS and ram corrupts the data before transit, then the transmitted file's checksum will not match the source's checksum. In either event, a retry will likely solve the issue as its unlikely for the same exact bytes of physical memory to be used over and over again
      - a1369209993 7y ago
        > the destination checksum will not match the source checksum There is no source checksum. > transmitted file's checksum will not match the source's checksum Most applications don't checksum their files when reading them. > a retry will likely solve the issue You can't retry; the only copy of the edited file is corrupt, and it's not possible to (automatically, in the general case) distinguish edits you made since loading the file out of ZFS from memory corruption that happened since loading the file out of ZFS.
- BubRoss 7y ago
  Just because ZFS isn't an exception doesn't mean ECC RAM is any less desirable for filesystems in general.
- tjoff 7y ago
  1. ZFS is heavy on memory, vastly increasing the odds that a failure will happen during filesystem operations. This is something many learnt while both encrypting their drives and overclocking their CPU. Encrypting your drives makes the CPU spend orders of magnitude more time processing filesystem data, and if an error is about to happen it is orders of magnitudes more likely you will have corrupt data because of it. Same goes with memory. 2. If an error occurs on ZFS the consequences are worse, because there are very limited recovery-tools available for ZFS. The official checkdisk is "just restore from tape, it's quicker than fsck anyway". Very enterprise oriented. Now you might be okay with the above. But the risk is greater and the potential consequences are worse. Not because of ZFS being special, but because of the way it makes use of your hardware. Also keep in mind that a lot of people pick ZFS to reduce potential corruption so the addition of ECC is par for the course.

blackflame7000 7y ago

I'm so sick of this argument every time a ZFS article comes up. This is by the inventor, Matthew Ahrens: “There’s nothing special about ZFS that requires/encourages the use of ECC RAM more so than any other filesystem.” https://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-your-data/ https://jrs-s.net/2015/02/03/will-zfs-and-non-ecc-ram-kill-y... Now give me my karma back.

dfeojm-zlib 7y ago
The thing about ECC vs. non-ECC. If you don't have ECC, you don't know silent bit corruption is happening, and when it's finally (if it's ever) discovered, there's nothing that can be done. ECC (SECDED) keeps data safer than it would be otherwise. ECC is necessary for anything real or important based on the financial cost of losing data. For everything else, it can be optional. Plus, don't forget the security issues of bitsquatting and other attacks that are the real results of silent bitflips. DEF CON 19 - Artem Dinaburg - Bit-squatting: DNS Hijacking Without Exploitation https://youtu.be/9WcHsT97suU https://youtu.be/9WcHsT97suU
- blackflame7000 7y ago
  All computers do error checking. Here is a summary: Non-ECC - Checksum: parity bit BlockSize: 1 byte ECC - Checksum: 7 bits BlockSize: 8 bytes ZFS - Checksum: 256 bits BlockSize: ~128kb
  - devonkim 7y ago
    It’s a lot further than this though with unbuffered RAM, registered RAM in the mix affecting performance and such. Error detection and error correction are different and require different levels of parity codes. Standard RAM at one point (SIMMs in the 90s) detected errors but won’t ever correct them. ECC can detect more but error patterns and now correct error patterns up to so many bits. The issues are more subtle beyond hardware though - your OS kernel has to understand the detection from the BIOS / EFI and if the motherboard manufacturer decided not to opt for wiring in the checksum fail signal you will run blind to data corruption that’s potentially really subtle on typical consumer hardware. With some BIOSes RAM failures result in a hard panic and will hard reboot the machine (the idea being a crash is better than continuing with an error). For me, ECC (both UDIMM and RDIMM) today are so inexpensive it’s a no-brainer for builds for business.
tasty_freeze 7y ago
The comment from the inventor isn't supporting your position. ZFS doens't need it any more, but if you care about data integrity, then you want ECC memory in your system. Yes, ZFS can take care of errors while it inside the ZFS API boundary, but the data on the outside of that is at risk, unless you have some higher level error correction protocol going on.
- blackflame7000 7y ago
  Don't make the mistake of thinking Non-ECC ram doesn't have built-in error protection it's just not as robust. Random memory bit flips are pretty rare but it takes multiple at the same time for ECC to be effective. Additionally, the thing that makes no sense about ECC Ram is that you have no parity checks when data is in transit too and from the CPU(s). So a little corrosion on a trace or bad capacitor and ECC Ram won't be able to do anything about it.
  - agapon 7y ago
    It's you who is making the mistake. Non-ECC RAM does not have any protection. Prove me wrong, describe the mechanism.
    - blackflame7000 7y ago
      https://en.wikipedia.org/wiki/RAM_parity https://en.wikipedia.org/wiki/RAM_parity
      - rincebrain 7y ago
        From that article: "The SDRAM and DDR modules that replaced the earlier types are usually available either without error-checking or with ECC (full correction, not just parity)."
        blackflame7000 7y ago
        "ECC (full correction, not just parity)" - meaning byte level parity is standard whereas a cache line checksum is extra.
a1369209993 7y ago
Can't speak for the downvoters, but I think what beatgammit is saying is that, in cases where you care about data corruption in ZFS, you also care about data corruption in parts of the system that aren't ZFS. Like the memory maps of every piece of application software. Nothing about ZFS needs ECC, but the circumstances ZFS claims to protect against need ECC.
- blackflame7000 7y ago
  No ZFS never protects against data in transit. It makes 1 promise, that the bytes written to disk will be the same bytes read from disk at a later point in time. ZFS can't be responsible for being given bad data. Corrupt data looks just as writable as normal data to a storage controller. The article I linked shows how even bad ram isn't damaging to the integrity of a properly configured ZFS file system.
- kissgyorgy 7y ago
  Exactly! You build a system with ZFS, because you need better defense against corruption, but without ECC, RAM will be the weak point in the system, so if you want the same level of guarantees, you need ECC, not because of ZFS.
  - blackflame7000 7y ago
    How so? You have a file from source A acquired via network, external storage, disc, etc and you want to store it on a ZFS array. If the ram corrupts the file before ZFS can store it, then the destination checksum will not match the source checksum. In reverse, you load the file from ZFS and ram corrupts the data before transit, then the transmitted file's checksum will not match the source's checksum. In either event, a retry will likely solve the issue as its unlikely for the same exact bytes of physical memory to be used over and over again
    - a1369209993 7y ago
      > the destination checksum will not match the source checksum There is no source checksum. > transmitted file's checksum will not match the source's checksum Most applications don't checksum their files when reading them. > a retry will likely solve the issue You can't retry; the only copy of the edited file is corrupt, and it's not possible to (automatically, in the general case) distinguish edits you made since loading the file out of ZFS from memory corruption that happened since loading the file out of ZFS.
BubRoss 7y ago
Just because ZFS isn't an exception doesn't mean ECC RAM is any less desirable for filesystems in general.
tjoff 7y ago
1. ZFS is heavy on memory, vastly increasing the odds that a failure will happen during filesystem operations. This is something many learnt while both encrypting their drives and overclocking their CPU. Encrypting your drives makes the CPU spend orders of magnitude more time processing filesystem data, and if an error is about to happen it is orders of magnitudes more likely you will have corrupt data because of it. Same goes with memory. 2. If an error occurs on ZFS the consequences are worse, because there are very limited recovery-tools available for ZFS. The official checkdisk is "just restore from tape, it's quicker than fsck anyway". Very enterprise oriented. Now you might be okay with the above. But the risk is greater and the potential consequences are worse. Not because of ZFS being special, but because of the way it makes use of your hardware. Also keep in mind that a lot of people pick ZFS to reduce potential corruption so the addition of ECC is par for the course.