3 ms·
Checksumming makes no sense on modern hardware. Hard Drives and SSDs use CRCs already. When a sector goes bad on a hard drive, the firmware will mess about ret
by cjensen 10y ago
Checksumming makes no sense on modern hardware. Hard Drives and SSDs use CRCs already.
When a sector goes bad on a hard drive, the firmware will mess about retrying and altering the analog amplifiers to try to get the signal back. If it gets the data back, it might "recover" by moving the data to spare sectors. Without a CRC, the firmware would have no way of differentiating between a sector read correctly and one that is read erroneously.
- prutschman 10y agoChecksumming makes sense when the checksum accompanies a pointer to another block. i.e. a directory entry says subdirectory contents are found in block 1027 /with checksum 0xdeadbeef/. If block 1027 is errantly overwritten due to a bug in the filesystem, the block driver, the DMA subsystem, or the device firmware, the FS will know as soon as it goes to fetch block 1027 that something went wrong. This is true even if the block is still internally consistent; perhaps the write was misdirected, another sector was incorrectly read, or a failure in the flash translation layer caused a newer write to be lost. If there's any redundancy to the system, whether storing the metadata in multiple places on the disk, or on different disks in an array, the FS can then check all the others and, crucially, detect which version is correct.
- ysleepy 10y agoWell it happens enough in practice, so it must come from somewhere. Maybe its the shitty SATA cable, or RAM bitflips, or the disk. I'll gladly spend 1% or whatever space overhead to notice it.
- mioelnir 10y agoSSDs use CRC to deliver their probabilistic data storage approximation. FS level checksums are, among others, for when that fails. FS level checksums also cover bitflips on your bus during the write phase (the drive will crc the already bad data) or during the read phase (you receive different data than your drive sent).
- nisa 10y agoAt least with bigger SATA disks I see regularly checksum errors on btrfs/ZFS and a rising bad sector count. The disk usually replaces the sectors on the next write but when reading it returns either zeros or wrong data. It's nice to know what files are affected from bad sectors. It's not common but common enough to have it.
- userbinator 10y agoWhat sort of "bigger SATA disks" are you using, or how are you treating them? If I see one bad sector develop on a disk for no obvious reason, it's getting replaced ASAP because it's a sign that more will follow very soon.
- nisa 10y agoseveral hundreds of 4TB HGST enterprise in a cluster at university. Disks get replaced ASAP but I'm seeing the checksum errors often in the logfiles when a disk starts to collapse. Btw: So far the Backblaze reliability numbers check out (<1% annual failure rate)