3 ms·

XFS has a long and solid track record, but without the workload-dependent regressions which can be encountered in a copy-on-write filesystem. Decent hard drive

XFS has a long and solid track record, but without the workload-dependent regressions which can be encountered in a copy-on-write filesystem. Decent hard drives have had internal checksums for ages. If the drive is even halfway functioning, it will catch the bad blocks--making that particular ZFS feature largely redundant, and databases that matter are going to be on a RAID anyway, which has its own additional layer of parity. ZFS is fantastic for some applications, not so much for others.

z3t4 4y ago
Ive had many checksum errors with ZFS, really no difference between enterprise Hdd or cheapest possible hdd (new ones). Or i just got unlucky. But some hdds are 15+ years, bad sectors but still usable.
- georgia_peach 4y ago
  Checksum vs parity. I would wager that the drive firmware also detected the error. This is why I think ZFS checksumming is largely redundant. Parity / error correction over a zpool being another issue entirely. This is not to ding ZFS. If I were in Sun's position of having to take responsibility for 3rd-party manufactured drives, I'd have done the same thing. And circa 2006, the reliability of in-drive error detection may have been in a poorer state. Never had a corruption issue the past 10 years that wasn't preceded by months of SMART errors/warnings.
  - tmikaeld 4y ago
    It is absolutely not redundant, ZFS RAIDZ can reconstruct damaged data, thanks to the extra checksum provided by ZFS. While traditional RAID does not recover from silent data corruption because it does not know which block to reconstruct. This is exceptionally sensitive when backups are encrypted, because even a few bits off and you can't decrypt the backups. Since this happens transparently, regularly testing backups are very important in non-ZFS systems.
    - georgia_peach 4y ago
      Read what I wrote more carefully. Checksumming is error detection only, while parity schemes can handle both error detection & correction. Checksumming blocks in software that are already being checksummed in hardware is redundant. Parity, on the other hand, is an essential part of any data redundancy system. For a transactional database, XFS + hardware RAID is going to be a more reliable setup than ZFS + JBOD. Database logs are similar in function to CoW, & the fancier DBs have time-machine features similar to snapshots. Having filesystem & software struggling against each other to solve the same problem, but with different solutions, is a situation that invites so many Heisenbugs. That, and a good RAID controller is going to handle drive-reported errors sooner and more gracefully than a ZFS scrub.

z3t4 4y ago

Ive had many checksum errors with ZFS, really no difference between enterprise Hdd or cheapest possible hdd (new ones). Or i just got unlucky. But some hdds are 15+ years, bad sectors but still usable.

georgia_peach 4y ago
Checksum vs parity. I would wager that the drive firmware also detected the error. This is why I think ZFS checksumming is largely redundant. Parity / error correction over a zpool being another issue entirely. This is not to ding ZFS. If I were in Sun's position of having to take responsibility for 3rd-party manufactured drives, I'd have done the same thing. And circa 2006, the reliability of in-drive error detection may have been in a poorer state. Never had a corruption issue the past 10 years that wasn't preceded by months of SMART errors/warnings.
- tmikaeld 4y ago
  It is absolutely not redundant, ZFS RAIDZ can reconstruct damaged data, thanks to the extra checksum provided by ZFS. While traditional RAID does not recover from silent data corruption because it does not know which block to reconstruct. This is exceptionally sensitive when backups are encrypted, because even a few bits off and you can't decrypt the backups. Since this happens transparently, regularly testing backups are very important in non-ZFS systems.
  - georgia_peach 4y ago
    Read what I wrote more carefully. Checksumming is error detection only, while parity schemes can handle both error detection & correction. Checksumming blocks in software that are already being checksummed in hardware is redundant. Parity, on the other hand, is an essential part of any data redundancy system. For a transactional database, XFS + hardware RAID is going to be a more reliable setup than ZFS + JBOD. Database logs are similar in function to CoW, & the fancier DBs have time-machine features similar to snapshots. Having filesystem & software struggling against each other to solve the same problem, but with different solutions, is a situation that invites so many Heisenbugs. That, and a good RAID controller is going to handle drive-reported errors sooner and more gracefully than a ZFS scrub.