7 ms·

Zfs has checksumming and scrubs, which can catch lots of data corruption that (most?) other filesystems can't catch catch at all

antongribok 3y ago
I guess I meant it as having a separate repair tool, but yes, you're of course correct. To me it was always weird to not have a separate tool for being able to do an offline ZFS repair. With regards to data corruption, I mean, this is exactly why I moved to btrfs, because it's able to catch bit rot same as ZFS. I think ultimately this is why ZFS data corruption bugs always are such a big deal... It's because in a lot of tech circles ZFS is put on this infallible pedestal where it can never do any harm to your data.
- AndrewDavis 3y ago
  Could you please explain what you mean by a separate repair tool kit?
  - lproven 3y ago
    I think he means an equivalent for `fsck`.
    - db48x 3y ago
      `zfs scrub` is equivalent to `fsck`. It just has a different name, and it can fix more kinds of corruption (including corrupted data, which `fsck` for other file systems can never do), and can do so on line. Of course zfs can fix those same errors on line while doing ordinary reads, so really all `zfs scrub` does is read everything. `zfs scrub` is better than `fsck`.
      - rincebrain 3y ago
        It is not. `zpool scrub` walks every block in the pool, and implicitly, you do checksums and other things while doing that. That's it. It's not doing any sort of logic bug repairs or cleanup or anything else. It's also not checking that you can, say, decrypt things, since that would mean you needed the keys to scrub.
        db48x 3y ago
        It really is better. The key difference to understand is that in ext4 any form of metadata corruption which can be automatically fixed requires you to take the filesystem off–line (to unmount it), and then run fsck. Meanwhile with zfs, any form of metadata corruption which can automatically be fixed is simply fixed right on the spot, transparently. In truth fsck is a wart, a kludge, a bag on the file system design.
        rincebrain 3y ago
        I am passingly acquainted with ZFS. The key thing to realize is that you can't, actually, automatically fix every problem, sometimes you have found a logic problem which results in an impossible outcome and you need someone to manually clean it up. In a world without flaws, it would be great to never need that. But the thing about theory and practice is that in theory, they never differ, but in practice...
        db48x 3y ago
        I know that. But fsck only fixes the ones that it can fix automatically. With anything else, you are completely on your own. Get a hex editor and go to town. With zfs, if there is some kind of problem that cannot be automatically fixed then at least you have one more tool available: zdb. It’s a _debugger_ for zfs filesystems. It will show you everything, more than you ever wanted to know. It is way better for fixing problems than any hex editor.
        rincebrain 3y ago
        zdb is read-only. It's not fixing anything, just telling you what's going on.
        db48x 3y ago
        Don’t be an idiot. You can fix more with zdb and a hex editor than you can with the hex editor alone.
- Volundr 3y ago
  > To me it was always weird to not have a separate tool for being able to do an offline ZFS repair. Have you ever asked yourself why those operations need to be done offline? It's not that ZFS doesn't have these capabilities, it's that ZFS is doing these similar checks and repairs as part of its normal operation and more. If you can't mount a ZFS filesystem, odds are a similarly borked ext4 filesystem isn't going to be repaired by fsck. I got bit by some pretty bad foot guns in BTRFS that has meant I steer pretty far away, but my understanding is these have been somewhat mitigated. I hope it continues to work well for you.

antongribok 3y ago

I guess I meant it as having a separate repair tool, but yes, you're of course correct. To me it was always weird to not have a separate tool for being able to do an offline ZFS repair. With regards to data corruption, I mean, this is exactly why I moved to btrfs, because it's able to catch bit rot same as ZFS. I think ultimately this is why ZFS data corruption bugs always are such a big deal... It's because in a lot of tech circles ZFS is put on this infallible pedestal where it can never do any harm to your data.

AndrewDavis 3y ago
Could you please explain what you mean by a separate repair tool kit?
- lproven 3y ago
  I think he means an equivalent for `fsck`.
  - db48x 3y ago
    `zfs scrub` is equivalent to `fsck`. It just has a different name, and it can fix more kinds of corruption (including corrupted data, which `fsck` for other file systems can never do), and can do so on line. Of course zfs can fix those same errors on line while doing ordinary reads, so really all `zfs scrub` does is read everything. `zfs scrub` is better than `fsck`.
    - rincebrain 3y ago
      It is not. `zpool scrub` walks every block in the pool, and implicitly, you do checksums and other things while doing that. That's it. It's not doing any sort of logic bug repairs or cleanup or anything else. It's also not checking that you can, say, decrypt things, since that would mean you needed the keys to scrub.
      - db48x 3y ago
        It really is better. The key difference to understand is that in ext4 any form of metadata corruption which can be automatically fixed requires you to take the filesystem off–line (to unmount it), and then run fsck. Meanwhile with zfs, any form of metadata corruption which can automatically be fixed is simply fixed right on the spot, transparently. In truth fsck is a wart, a kludge, a bag on the file system design.
        rincebrain 3y ago
        I am passingly acquainted with ZFS. The key thing to realize is that you can't, actually, automatically fix every problem, sometimes you have found a logic problem which results in an impossible outcome and you need someone to manually clean it up. In a world without flaws, it would be great to never need that. But the thing about theory and practice is that in theory, they never differ, but in practice...
        db48x 3y ago
        I know that. But fsck only fixes the ones that it can fix automatically. With anything else, you are completely on your own. Get a hex editor and go to town. With zfs, if there is some kind of problem that cannot be automatically fixed then at least you have one more tool available: zdb. It’s a _debugger_ for zfs filesystems. It will show you everything, more than you ever wanted to know. It is way better for fixing problems than any hex editor.
        rincebrain 3y ago
        zdb is read-only. It's not fixing anything, just telling you what's going on.
        db48x 3y ago
        Don’t be an idiot. You can fix more with zdb and a hex editor than you can with the hex editor alone.
Volundr 3y ago
> To me it was always weird to not have a separate tool for being able to do an offline ZFS repair. Have you ever asked yourself why those operations need to be done offline? It's not that ZFS doesn't have these capabilities, it's that ZFS is doing these similar checks and repairs as part of its normal operation and more. If you can't mount a ZFS filesystem, odds are a similarly borked ext4 filesystem isn't going to be repaired by fsck. I got bit by some pretty bad foot guns in BTRFS that has meant I steer pretty far away, but my understanding is these have been somewhat mitigated. I hope it continues to work well for you.