4 ms·

My understanding is that ZFS does a lot of healing itself on the fly. When you access a block, the block is checked against a checksum stored elsewhere on disk.

by jiggy2011 14y ago

My understanding is that ZFS does a lot of healing itself on the fly. When you access a block, the block is checked against a checksum stored elsewhere on disk. If the block is bad, it is restored from a replica (so it's advisable to have 2 disks) while the original is healed. On writes ZFS also checks that each written block is valid before committing the new block to the filesystem. In the event of a bad write the new block is not committed and the old block remains, blocks are written originally to a separate area of the disk.

joosters 14y ago
Yes, and those are good features. But there's still plenty of room for errors. Bugs in the code, bad interactions with hardware, etc. Those are areas where ZFS is likely to be just as vulnerable as other file systems. More so, in fact, given the code size/complexity and its (relative) immaturity. I suspect (but I have no data!) that the majority of times that users have file system corruption and need to run fsck is due to bad software behaviour, and not failing disks.
- jiggy2011 14y ago
  ZFS itself has been around for a while, was originally part of Solaris IIRC. It's using it on Linux that's new. If you have a bad software write, ZFS should be able to detect that in the same way it would a hardware error since the checksums won't match. Also worth noting that the checksums are themselves checksummed in a merkle tree. It also has a tool called "scrub" which can check data integrity without unmounting the disk. Of course if your filesystem has buggy code for checksumming/repair then you are boned whatever.
- dmpk2k 14y ago
  Disk lie. A lot. Ask anybody who has the misfortune to work on a filesystem for any length of time. One thing to keep in mind is that ZFS is more complex than your typical filesystem, but it's also one of the few filesystems that's effectively a cryptographically self-validating tree. You're a lot more likely to have a disk lie to you than ZFS flake out.
  - tjoff 14y ago
    And the whole point of RAID is to combat that... But since there is no decent recovery tools for ZFS your entire FS might be toast if the wrong bits are turned. Yes ZFS is quite resilient against corruption but when it bites, which has happened plenty of times in the past, you are truly screwed. The philosophy behind ZFS and file system recovery has always been that it is too inconvenient. It is much better to restore from tape backups so why make any effort on recovery? Thing is, most consumers don't have tape backups... Because of that, ZFS is in the very vast majority of cases a bad choice for home users, especially under linux where it can hardly be considered mature. BRTFS just isn't ready yet. As a home user you can't even get a decent COW FS with decent snapshotting or checksums. That is the sad state we are in and will be in for many years to come.
    - jiggy2011 14y ago
      I don't quite understand what corruption scenarios fsck on Ext4 would save you from that ZFS wouldn't? Most data on consumer's computers is 1 disk failure away from loss anyway, hence the popularity of cloud syncing services.
    - X-Istence 14y ago
      Please tell me how my entire FS will be toast if the wrong bits are turned, especially when zraid2 is really the only valid solution that should be used. I've been using ZFS for years now, i've had disks go bad, I've replaced them. ZFS has protected me from data loss better than anything else. In the past when I've had the disk lie to me, ZFS was able to tell me what files were corrupted and thus had invalid data. I'm going to need to throw a [citation needed] on your post.
      - tjoff 14y ago
        If this is news to you you really haven't been paying attention... http://www.osnews.com/story/22423/Should_ZFS_Have_a_fsck_Tool_ http://www.osnews.com/story/22423/Should_ZFS_Have_a_fsck_Too...
        14y ago

joosters 14y ago

Yes, and those are good features. But there's still plenty of room for errors. Bugs in the code, bad interactions with hardware, etc. Those are areas where ZFS is likely to be just as vulnerable as other file systems. More so, in fact, given the code size/complexity and its (relative) immaturity. I suspect (but I have no data!) that the majority of times that users have file system corruption and need to run fsck is due to bad software behaviour, and not failing disks.

jiggy2011 14y ago
ZFS itself has been around for a while, was originally part of Solaris IIRC. It's using it on Linux that's new. If you have a bad software write, ZFS should be able to detect that in the same way it would a hardware error since the checksums won't match. Also worth noting that the checksums are themselves checksummed in a merkle tree. It also has a tool called "scrub" which can check data integrity without unmounting the disk. Of course if your filesystem has buggy code for checksumming/repair then you are boned whatever.
dmpk2k 14y ago
Disk lie. A lot. Ask anybody who has the misfortune to work on a filesystem for any length of time. One thing to keep in mind is that ZFS is more complex than your typical filesystem, but it's also one of the few filesystems that's effectively a cryptographically self-validating tree. You're a lot more likely to have a disk lie to you than ZFS flake out.
- tjoff 14y ago
  And the whole point of RAID is to combat that... But since there is no decent recovery tools for ZFS your entire FS might be toast if the wrong bits are turned. Yes ZFS is quite resilient against corruption but when it bites, which has happened plenty of times in the past, you are truly screwed. The philosophy behind ZFS and file system recovery has always been that it is too inconvenient. It is much better to restore from tape backups so why make any effort on recovery? Thing is, most consumers don't have tape backups... Because of that, ZFS is in the very vast majority of cases a bad choice for home users, especially under linux where it can hardly be considered mature. BRTFS just isn't ready yet. As a home user you can't even get a decent COW FS with decent snapshotting or checksums. That is the sad state we are in and will be in for many years to come.
  - jiggy2011 14y ago
    I don't quite understand what corruption scenarios fsck on Ext4 would save you from that ZFS wouldn't? Most data on consumer's computers is 1 disk failure away from loss anyway, hence the popularity of cloud syncing services.
  - X-Istence 14y ago
    Please tell me how my entire FS will be toast if the wrong bits are turned, especially when zraid2 is really the only valid solution that should be used. I've been using ZFS for years now, i've had disks go bad, I've replaced them. ZFS has protected me from data loss better than anything else. In the past when I've had the disk lie to me, ZFS was able to tell me what files were corrupted and thus had invalid data. I'm going to need to throw a [citation needed] on your post.
    - tjoff 14y ago
      If this is news to you you really haven't been paying attention... http://www.osnews.com/story/22423/Should_ZFS_Have_a_fsck_Tool_ http://www.osnews.com/story/22423/Should_ZFS_Have_a_fsck_Too...
      - 14y ago