4 ms·
How? Bitrot occurs because the lower-abstraction level hardware fails. When you put a Hard-Drive into storage for say 5 years, the bits may change. Even if a H
by dragontamer 9y ago
How?
Bitrot occurs because the lower-abstraction level hardware fails. When you put a Hard-Drive into storage for say 5 years, the bits may change. Even if a Hard-drive remains in constant use for 5 years... if said files or directories aren't checked and double-checked constantly, the error-correction codes may fail over time.
Its a fundamentally different problem from Hard Drives that are being used constantly as say Swap.
Hard Drives typically include Hamming codes or ECC bits to address typical corruption issues.
-------------
The fundamental principle at hand here is as follows: to ensure integrity of files, you need to regularly check file data. Only the Filesystem would know which files were recently checked.
- Koshkin 9y agoCouldn't the drive's firmware or the driver do the same just as well (except on physical records instead of files)?
- dragontamer 9y agoFirst off, real hard drives have "SMART" data that detects (and automatically corrects) simple errors. So remember, hard drives ALREADY have a large degree of error correction built in. Its just not enough for serious data-storage purposes. The "Bit-Rot" scenario is particularly harmful to RAID5 (Minimum 3-hard drives. Two contain data, one contains "parity" that can fix any errors on the other hard drives. Then the parity is structured to be striped equally across the three drives). Modern RAID drivers can do this rather easily. The problem with "Bit Rot" is that a RAID5 drive will not rebuild itself until it detects an error. If you're reading files along, and all of a sudden... the hard drive detects an error. No problem (in the typical case), just rebuild the data from the parity. However, "Bit Rot" means that the parity bits (on the 3rd backup hard drive) have ALSO rotted away. ---------- The only way to fix this "bit-rot" error is to constantly read through your data and CONSTANTLY check for bit-rot. No hard drive is going to silently spin and hamper-performance of the system for self-verification purposes... but a Filesystem / Operating system can schedule these "Scrubs" to occur during periods of low-I/O. Which is how ZFS, and Window's ReFS work. When your computer is idle, the OS checks for bitrot. When the computer starts to work again, it pauses the "low priority" bitrot checks and serves the data. ------- ZFS doesn't quite work like Window's ReFS. ZFS simply checks for bit rot whenever a file is accessed. Every time. There are "ZFS Scrub" commands (which you can put into a cron-job) to read every file (and therefore check for bitrot).
- KaiserPro 9y ago"enterprise" storage arrays do this for you. You have a glob of storage that is presented as a block device, but actually underneath is a reas no fooling filesystem with FEC, Snapshots and all sorts of other goodies.