3 ms·
Is there something like Aphyr's Jespen for file systems? https://aphyr.com/tags/Jepsen https://aphyr.com/tags/Jepsen I've been curious about ZFS, btrfs, etc.
by specialist 10y ago
Is there something like Aphyr's Jespen for file systems?
https://aphyr.com/tags/Jepsen https://aphyr.com/tags/Jepsen
I've been curious about ZFS, btrfs, etc. But as a layperson, I don't have the technical chops, gumption, wherewithal to figure what's what. Reading posts (comments) about the edge cases where they fail (data, performance, missing features) leaves me more baffled.
- ryao 10y agoThere is plenty there to read. I would rather not spend my Sunday reading that to see if I can think of something equivalent. However, you might find these pages helpful: http://open-zfs.org/wiki/System_Administration http://open-zfs.org/wiki/System_Administration http://open-zfs.org/wiki/Performance_tuning http://open-zfs.org/wiki/Performance_tuning http://open-zfs.org/wiki/Hardware http://open-zfs.org/wiki/Hardware There are also mailing lists and IRC channels where you can ask questions. On freenode, there are #zfsonlinux, #openzfs, #openzfs-osx and #zfs.
- specialist 10y agoSorry, I didn't mean for you to read it, just linked to be clear. The Jespen effort verifies the reliability claims of various persistence schemes. Independent verification, if you will.
- ryao 10y agoZFS has a utility called ztest that performs stochastic testing on the core ZFS code in userspace with assertions enabled. It catches many reliability issues and is something that anyone can run. There is also the zinject framework for doing fault injection. As for independent verification, here are the top two results from searching google from my IP address for "Zfs paper reliability": http://pages.cs.wisc.edu/~kadav/zfs/zfsrel.pdf http://pages.cs.wisc.edu/~kadav/zfs/zfsrel.pdf http://research.cs.wisc.edu/adsl/Publications/zfs-corruption-fast10.pdf http://research.cs.wisc.edu/adsl/Publications/zfs-corruption... There is nothing surprising to me there. The merkle tree with good quality 256-bit checksums strongly guarantees that corruptions are detected and redundancy allows for correction. The main ways of messing up would be a buggy driver, something else in the kernel damaging the driver's data structures or hardware corrupting memory. The stochastic testing gives some protection against the first of those. I guess you could target the two things that live outside of the merkle tree (labels and ZIL blocks), but those are self checksummed and used in ways that minimize the potential for issues. Here is a summary of data integrity features that I wrote: http://open-zfs.org/w/index.php?title=System_Administration#Data_Integrity http://open-zfs.org/w/index.php?title=System_Administration#... It is possible that I missed some of the more obscure protections.
- ymse 10y agoNot the same, but there was an article about file system fuzzing recently: https://news.ycombinator.com/item?id=11469535 https://news.ycombinator.com/item?id=11469535 The presentation is focused on how the fuzzing technique works rather than individual file system performance, but the "time to first bug" is quite telling. Copied here: ext4 (2h) XFS (1h45) GFS2 (8m) NTFS (4m) NILFS2 (1m) HFS (30s) HFS+ (25s) ReiserFS (25s) OCFS2 (15s) F2FS (10s) BTRFS (5s) I've had miscellaneous failures with the bottom three, but XFS and ext4 have been rock solid. Unfortunately ZFS and CephFS were not included. Would love to see a Jepsen-style analysis of the various file systems too.
- ryao 10y agoThat sort of analysis is testing something on most of those that happen to be two different things on btrfs and ZFS. Specifically, corruption that passes checksums and corruption that does not pass checksums. btrfs' result was obtained by modifying it to disable checksums. How btrfs or ZFS do without the benefit of checksums should only matter for people exchanging disk images. In that case, it would be better to have a userland driver regardless of the filesystem. NetBSD is able to run its filesystem drivers in userspace for this reason.