4 ms·
> > Filesystem-level de-duplication is scary as hell as a concept > What's scary about it? It's scary because there's only one copy when you might have expect
by cesarb 3y ago
> > Filesystem-level de-duplication is scary as hell as a concept
> What's scary about it?
It's scary because there's only one copy when you might have expected two. A single bad block could lose both "copies" at once.
- ForkMeOnTinder 3y agoCopying a file isn't great protection against bad blocks. Modern SSDs, when they fail, tend to fail catastrophically (the whole device dies all at once), rather than dying one block at a time. If you care about the data, back it up on a separate piece of hardware.
- grepfru_it 3y agoThe file system metadata is redundant and on a correctly configured ZFS system your error correction is isolated and can be redundant as well
- Filligree 3y agoDisks die all the time anyway. If you want to keep your data, you should have at least two-disk redundancy. In which case bad blocks won't kill anything.
- phpisthebest 3y ago3-2-1.. 3 Copies 2 Media 1 offsite. If you follow that then you would have no fear of data loss. if you are putting 2 copies on the same filesystem you are already doing backups wrongs
- modderation 3y agoI always like to add: 0 stress restore. You should also be able to restore your data in a calm, controlled, and correct manner. Test your backups to be sure they work, and to be sure that you're still familiar with the process. You don't want to be stuck reverse-engineering your backup solution in the middle of an already-panicky data loss scenario. Remain calm, follow the prescribed steps, and wait for your data to come back.
- magicalhippo 3y agoBesides what all the others have mentioned, you can force ZFS to keep up to 3 copies of data blocks on a dataset. ZFS uses this internally for important metadata and will try to spread them around to maximize the chance of recovery, though don't rely on this feature alone for redundancy.
- stormking 3y agoThat's what data checksumming and mirroring is for.
- opk 3y agoYou can do `zfs set copies=2` for a ZFS dataset if you think there's value in the extra copies. That's better than a copy of the file because with a single bad block, the checksum will fail and ZFS will retrieve the data from the good block and repair both files. In practice, using disk mirrors is better than setting the copies property.