3 ms·

> I hear and read conflicting testimonies about ZFS which supposedly handles data corruption better and will give it a try at some point. For now, I use md5sum

> I hear and read conflicting testimonies about ZFS which supposedly handles data corruption better and will give it a try at some point. For now, I use md5sum to create and verify checksums of my backup .tar.gz archives and parchive to create error recovery files. I’ve written about parchive before and remain to this day a dedicated fan. I’m very surprised you went through the trouble of building such an amazing NAS and skipped ZFS! Definitely give it a try. It’s especially useful for combatting disk rot. Not to mention how easy it is to setup and use.

selfhoster11 6y ago
ZFS adds complexity and licensing issues into the mix. parchive is a good way to separate different layers of concerns to achieve similar end point.
julianwachholz 6y ago
It definitely sounds overwhelming at first but once you get the gist of it it is quite easy to manage. I even somehow managed to migrate my FreeBSD pool to OpenZFS without any major hurdles.
ggeorgovassilis 6y ago
Thanks. I haven't tried ZFS so far for historical reasons: the post describes the latest evolution of my NAS; in fact, there were several hardware versions before with the hard disks being transplanted (and later upgraded) from the previous to the next. The filesystem structures are over a decade old, having been upgraded from ext2 to ext3 and now ext4. The next thing I will try out is dm-integrity.
- electricant 6y ago
  Why not also have a look at btrfs?
- pronoiac 6y ago
  I bookmarked a ZFS-inspired storage system for preserving data integrity, and dm-integrity has a role there: https://insanity.industries/post/preserving-data-integrity/ https://insanity.industries/post/preserving-data-integrity/ Hacker News discussion: https://news.ycombinator.com/item?id=24372662 https://news.ycombinator.com/item?id=24372662
MayeulC 6y ago
Most filesystems that are designated as "modern" include checksumming, snapshots, cow, etc. However, do some include support for CRC? That would allow not only detecting, but also correcting bitrot. I assume that's what Raid6 does, albeit less granular than per file, and more aimed at recovering a full file than a few bits? Can anyone point me to a FS that supports it, or to the reason it's not done? As for ZFS, I think I'll keep avoiding it until it is mainlined. I don't really want to play that game.

selfhoster11 6y ago

ZFS adds complexity and licensing issues into the mix. parchive is a good way to separate different layers of concerns to achieve similar end point.

julianwachholz 6y ago

It definitely sounds overwhelming at first but once you get the gist of it it is quite easy to manage. I even somehow managed to migrate my FreeBSD pool to OpenZFS without any major hurdles.

ggeorgovassilis 6y ago

Thanks. I haven't tried ZFS so far for historical reasons: the post describes the latest evolution of my NAS; in fact, there were several hardware versions before with the hard disks being transplanted (and later upgraded) from the previous to the next. The filesystem structures are over a decade old, having been upgraded from ext2 to ext3 and now ext4. The next thing I will try out is dm-integrity.

electricant 6y ago
Why not also have a look at btrfs?
pronoiac 6y ago
I bookmarked a ZFS-inspired storage system for preserving data integrity, and dm-integrity has a role there: https://insanity.industries/post/preserving-data-integrity/ https://insanity.industries/post/preserving-data-integrity/ Hacker News discussion: https://news.ycombinator.com/item?id=24372662 https://news.ycombinator.com/item?id=24372662

MayeulC 6y ago

Most filesystems that are designated as "modern" include checksumming, snapshots, cow, etc. However, do some include support for CRC? That would allow not only detecting, but also correcting bitrot. I assume that's what Raid6 does, albeit less granular than per file, and more aimed at recovering a full file than a few bits? Can anyone point me to a FS that supports it, or to the reason it's not done? As for ZFS, I think I'll keep avoiding it until it is mainlined. I don't really want to play that game.