3 ms·

While it is possible to have an unimportable pool, it is also possible to have ext4 and XFS filesystems that can neither be mounted nor repaired with fsck. When

by ryao 12y ago

While it is possible to have an unimportable pool, it is also possible to have ext4 and XFS filesystems that can neither be mounted nor repaired with fsck. When dealing with undefined behavior caused by bit flips, virtually any failure is possible, especially when you consider bit flips to kernel data structures. No software can save you from bit flips and if they are your concern (as they should be), then you should refuse to use computers that lack ECC RAM. As for the articles that you link, they are correct to say that you want to use ECC RAM. However, there is nothing specific to ZFS that makes it require ECC any more than any other filesystem. It should also be noted that I wrote the official documentation on this subject. It can be found at the Open ZFS wiki, rather than the pages you linked: http://open-zfs.org/wiki/Hardware#ECC_Memory http://open-zfs.org/wiki/Hardware#ECC_Memory

moe 12y ago
I apologize. I was sure to be dealing with a troll and was sloppy with my sources (i.e. I was sure to have once read it in the official docs, but here I just linked the first official looking google hit that was not the solaris docs). there is nothing specific to ZFS that makes it require ECC any more than any other filesystem This statement still troubles me. I was under the assumption that ZFS scrubbing may indeed lead to compounding corruption as described in the blog post I linked. Hence the ongoing(?) debate in the ZFS community. However, I consider myself disqualified from this discussion and sorry again for my tone. I'll read up on the current state of the argument.
- ryao 12y ago
  It is easy to get confused and it could happen to anyone. That said, the only thing in the code remotely similar to "compound corruption" is logic for preventing such a scenario, which has the following comment: /* * Don't rewrite known good children. * Not only is it unnecessary, it could * actually be harmful: if the system lost * power while rewriting the only good copy, * there would be no good copies left! */ https://github.com/zfsonlinux/zfs/blob/master/module/zfs/vdev_mirror.c#L481 https://github.com/zfsonlinux/zfs/blob/master/module/zfs/vde... That touches more than obvious at a glance because the mirror code is used for processing both mirrors and ditto blocks.
  - anon1385 12y ago
    Sorry for the very late reply that you probably won't see. I'm not familiar with ZFS at all, so I don't know what the terminology 'ditto blocks' and 'mirrors' mean, so there's a good chance I'm not talking sense here at all. The main argument for ZFS on non-ECC using systems being worse than other less sophisticated file systems seems to be that just reading a file (or during periodic 'scrubbing') while you have bad ram can cause good files to be corrupted when ZFS notices checksum errors and tries to correct them. (There are other arguments about not being able to mount pools due to bad ram, but that seems no different to any other FS). People seem to be quite insistent that this is possible, and at first glance the code you linked above doesn't seem to be relevant since it refers to 'known good' data, but the problem scenario is when data has (erroneously) been determined to be bad due to faulty ram. Will ZFS overwrite good data in that situation in an attempt to correct what it sees as a disk error? A few links where people are making this claim: https://forums.freenas.org/index.php?threads/ecc-vs-non-ecc-ram-and-zfs.15449/page-5#post-90394 https://forums.freenas.org/index.php?threads/ecc-vs-non-ecc-... https://forums.freenas.org/index.php?threads/ecc-vs-non-ecc-ram-and-zfs.15449/page-8#post-103799 https://forums.freenas.org/index.php?threads/ecc-vs-non-ecc-...

by ryao 12y ago

moe 12y ago

I apologize. I was sure to be dealing with a troll and was sloppy with my sources (i.e. I was sure to have once read it in the official docs, but here I just linked the first official looking google hit that was not the solaris docs). there is nothing specific to ZFS that makes it require ECC any more than any other filesystem This statement still troubles me. I was under the assumption that ZFS scrubbing may indeed lead to compounding corruption as described in the blog post I linked. Hence the ongoing(?) debate in the ZFS community. However, I consider myself disqualified from this discussion and sorry again for my tone. I'll read up on the current state of the argument.

ryao 12y ago
It is easy to get confused and it could happen to anyone. That said, the only thing in the code remotely similar to "compound corruption" is logic for preventing such a scenario, which has the following comment: /* * Don't rewrite known good children. * Not only is it unnecessary, it could * actually be harmful: if the system lost * power while rewriting the only good copy, * there would be no good copies left! */ https://github.com/zfsonlinux/zfs/blob/master/module/zfs/vdev_mirror.c#L481 https://github.com/zfsonlinux/zfs/blob/master/module/zfs/vde... That touches more than obvious at a glance because the mirror code is used for processing both mirrors and ditto blocks.
- anon1385 12y ago
  Sorry for the very late reply that you probably won't see. I'm not familiar with ZFS at all, so I don't know what the terminology 'ditto blocks' and 'mirrors' mean, so there's a good chance I'm not talking sense here at all. The main argument for ZFS on non-ECC using systems being worse than other less sophisticated file systems seems to be that just reading a file (or during periodic 'scrubbing') while you have bad ram can cause good files to be corrupted when ZFS notices checksum errors and tries to correct them. (There are other arguments about not being able to mount pools due to bad ram, but that seems no different to any other FS). People seem to be quite insistent that this is possible, and at first glance the code you linked above doesn't seem to be relevant since it refers to 'known good' data, but the problem scenario is when data has (erroneously) been determined to be bad due to faulty ram. Will ZFS overwrite good data in that situation in an attempt to correct what it sees as a disk error? A few links where people are making this claim: https://forums.freenas.org/index.php?threads/ecc-vs-non-ecc-ram-and-zfs.15449/page-5#post-90394 https://forums.freenas.org/index.php?threads/ecc-vs-non-ecc-... https://forums.freenas.org/index.php?threads/ecc-vs-non-ecc-ram-and-zfs.15449/page-8#post-103799 https://forums.freenas.org/index.php?threads/ecc-vs-non-ecc-...