2 ms·

Checksum vs parity. I would wager that the drive firmware also detected the error. This is why I think ZFS checksumming is largely redundant. Parity / error

Checksum vs parity. I would wager that the drive firmware also detected the error. This is why I think ZFS checksumming is largely redundant. Parity / error correction over a zpool being another issue entirely. This is not to ding ZFS. If I were in Sun's position of having to take responsibility for 3rd-party manufactured drives, I'd have done the same thing. And circa 2006, the reliability of in-drive error detection may have been in a poorer state. Never had a corruption issue the past 10 years that wasn't preceded by months of SMART errors/warnings.

tmikaeld 4y ago
It is absolutely not redundant, ZFS RAIDZ can reconstruct damaged data, thanks to the extra checksum provided by ZFS. While traditional RAID does not recover from silent data corruption because it does not know which block to reconstruct. This is exceptionally sensitive when backups are encrypted, because even a few bits off and you can't decrypt the backups. Since this happens transparently, regularly testing backups are very important in non-ZFS systems.
- georgia_peach 4y ago
  Read what I wrote more carefully. Checksumming is error detection only, while parity schemes can handle both error detection & correction. Checksumming blocks in software that are already being checksummed in hardware is redundant. Parity, on the other hand, is an essential part of any data redundancy system. For a transactional database, XFS + hardware RAID is going to be a more reliable setup than ZFS + JBOD. Database logs are similar in function to CoW, & the fancier DBs have time-machine features similar to snapshots. Having filesystem & software struggling against each other to solve the same problem, but with different solutions, is a situation that invites so many Heisenbugs. That, and a good RAID controller is going to handle drive-reported errors sooner and more gracefully than a ZFS scrub.

tmikaeld 4y ago

It is absolutely not redundant, ZFS RAIDZ can reconstruct damaged data, thanks to the extra checksum provided by ZFS. While traditional RAID does not recover from silent data corruption because it does not know which block to reconstruct. This is exceptionally sensitive when backups are encrypted, because even a few bits off and you can't decrypt the backups. Since this happens transparently, regularly testing backups are very important in non-ZFS systems.

georgia_peach 4y ago
Read what I wrote more carefully. Checksumming is error detection only, while parity schemes can handle both error detection & correction. Checksumming blocks in software that are already being checksummed in hardware is redundant. Parity, on the other hand, is an essential part of any data redundancy system. For a transactional database, XFS + hardware RAID is going to be a more reliable setup than ZFS + JBOD. Database logs are similar in function to CoW, & the fancier DBs have time-machine features similar to snapshots. Having filesystem & software struggling against each other to solve the same problem, but with different solutions, is a situation that invites so many Heisenbugs. That, and a good RAID controller is going to handle drive-reported errors sooner and more gracefully than a ZFS scrub.