3 ms·

From https://orib.dev/gefs.pdf https://orib.dev/gefs.pdf - > While snapshot consistency is useful to keep data consistent, disks often fail over time. In order

From https://orib.dev/gefs.pdf https://orib.dev/gefs.pdf - > While snapshot consistency is useful to keep data consistent, disks often fail over time. In order to detect corruption, block pointers contain a hash of the data that they point at. If corrupted data is returned by the underlying storage medium, this is detected via block hashes. And if a programmer error causes the file system to write garbage to disk, this can often be caught early. The corruption is reported, and the damaged data may then be recovered from backups, RAID restoration, or some other means. Okay! It's got CoW, snapshots, and data checksums. Therefore, it's good enough to compete with ZFS while being way smaller and permissively licensed. Now I just want it ported to Linux and the other BSDs:)

atmosx 11d ago
I don't think it will compete with ZFS or BTRFS (e.g. I don't think ppl will use GEFS over ZFS or BTRFS for a storage server), but it's a modern, much needed FFS replacement.
- yjftsjthsd-h 11d ago
  Who said anything about storage servers? I'm using zfs on laptops and desktops right now because I want data checksums and a filesystem that doesn't have a history of breaking horribly (I dropped btrfs after the second time it hosed my rootfs). Given the license issue with zfs - and in particular, the technical fallout like needing dkms - I'd be very pleased to replace it.
  - sippingabonedry 11d ago
    > I dropped btrfs after the second time it hosed my rootfs btrfs fans use the "you're using it wrong" excuse a lot. I recall a failure mode that activated when you fill the FS to 100% and their response was "you should never fill a filesystem to capacity"
    - jeffrallen 11d ago
      Otoh, good luck bringing a CoW filesystem back from 100%. Delete a file? Sure, let me just make a copy of all the metadata that was pointing at it using... the zero blocks I have left. Tradeoffs are a bitch, bitch.
      - yjftsjthsd-h 11d ago
        Which is why ZFS reserves "slop space" to make sure that doesn't happen, instead of defaulting to making it easy for users to corner themselves like that.
    - thetallguyyy 10d ago
      It's really a shame, though. btrfs is permanently unfinished, and zfs has the wrong license. End result is people stick with ext4 or xfs most of the time anyway.
    - scheme271 10d ago
      That failure mode is still there although it's been mitigated a bit. I hit it a few months ago but was able to recover after a few hours of doing various stuff.
      - sippingabonedry 10d ago
        Yeah that's not how filesystems are supposed to work. You should be able to dd /dev/zero to a file and not worry about the entire filesystem shitting the bed.
    - crest 10d ago
      They also fucked up their parity RAID repair so that the next disk failure on a different disk will destroy the data, argued that the pathological B-tree performance reported by a user couldn't be real, because they didn't understood the tree depth would only be logarithmic to the base of the fanout if the entries are the same size. insert shocked pikachu face
  - whalesalad 11d ago
    I have been hearing noise recently that btrfs is risky and unstable but (knocks on wood) i've been running it for years now with zero issues. What am I missing?
    - crabbone 11d ago
      Failures in storage software are very rare. Which makes it very hard to test... (you need to run it a lot, for a very long time if you hope to find errors by chance). Also, some failure modes are worse than others. The failures known as DI (data integrity) are the worst. Even though they aren't expected to happen to everyone at a certain frequency (because, again, mature storage software is comparatively very reliable), even a single DI error that happened to any user sets up a major alarm. In the storage industry, the running joke is that after first DI in your product you lose funding, after the second DI you loose the product. And it did happen to Btrfs quite a bit... I've seen it with my own eyes when a system didn't come back after power failure. (But I'm in the business of testing software storage products, so, it's less surprising that it happened to me). So... it's perfectly plausible that you have never seen Btrfs fail, and it's been more error prone than eg. EXT4. The error rate is low enough so that if you don't actively try to cause the error you will never experience one. But, over a large group of diverse use patterns, the rate is still worse than expected.
      - hulitu 9d ago
        > Failures in storage software are very rare. Famous last words. Most failures are not reported, because people move on.
    - gucci-on-fleek 11d ago
      btrfs is almost 20 years old now, so lots of people only used it back when it was newer and far buggier. In my experience, btrfs is actually more reliable than other filesystems due to its checksumming abilities, but when it does fail, it's much harder to fix than with other filesystems (which will often try to continue on even when stuff is broken).
      - deleted 10d ago
  - atmosx 11d ago
    > Who said anything about storage servers? I did. > I'm using zfs [...] ZFS is primarily used on single-storage appliances.
- gigatexal 10d ago
  It’ll replace brtfs if it gets ported. Almost anything is better than that pile of garbage.
  - LargoLasskhyfv 10d ago
    Running it for more than two years now, without any hassles. On single SSDs though. But still, waddya even talkin about?
    - cestith 10d ago
      When I last took a serious look at btrfs its RAID 5 and RAID 6 modes could lose a whole lot of data if a single drive crashed or if the system lost power during a write. That's not supposed to happen on a RAID system with a journal. From what I understand single copy setups, RAID 1, and RAID 10 never had the same problem. It's left a bad taste for a lot of people though. Very early ext4 versions also had some data loss or data corruption, but I never hear people dismiss it completely for that brief spot of its history. Maybe part of that is because it was around 17 years ago and it's been a reliable default OS for many years now.
      - scheme271 10d ago
        The recommendation is still to store metadata on a raid 1 or 1/0 partition and the data on a raid 5/6. Apparently there's still some issues with raid 5/6
        cestith 8d ago
        That's good information to have. Thanks.
    - throw0101a 9d ago
      > Running it for more than two years now, without any hassles. On single SSDs though. But still, waddya even talkin about? Running ZFS since it was originally released on Solaris (June 2006) with zero problems (then FreeBSD, then Linux), including on RAID-5/6 (RAID-Z1/2) configurations, which btrfs still does not recommend using all these years later: * https://btrfs.readthedocs.io/en/latest/btrfs-man5.html#raid56-status-and-recommended-practices https://btrfs.readthedocs.io/en/latest/btrfs-man5.html#raid5... What have the btrfs developer(s) been doing for the last decade(+) that the code is still of debatable dependability?
mmooss 11d ago
I've always wondered about similar designs: Doesn't calculating a hash of every block, on every read and every write, create lots of overhead? Why isn't that a problem? Some systems have dedicated crypto co-processors for confidentiality (encryption) - e.g., I think drives with FDE, and I think Apple Silicon SoCs might have them. Can those be repurposed for hash calculation? What about systems that lack them?
- chasil 11d ago
  Both ZFS and modern btrfs support a large set of checksums. Both implement sha256, which does impose a heavy speed penalty. ZFS allows you to adjust the checksum on the fly, using something faster (Fletcher) if desired. In btrfs, a global checksum is set at filesystem creation; xxhash is the best modern option. There is a website: https://xxhash.com https://xxhash.com Deduplication adds concerns for a strong hash free of collisions.
  - ThePowerOfFuet 11d ago
    Fletcher has been the default for quite some time.
    - throw0101a 10d ago
      Specifically "fletcher4": * https://openzfs.github.io/openzfs-docs/man/master/7/zfsprops.7.html#checksum https://openzfs.github.io/openzfs-docs/man/master/7/zfsprops... * https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Data%20Storage/Checksums.html https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Data... * https://en.wikipedia.org/wiki/Fletcher%27s_checksum https://en.wikipedia.org/wiki/Fletcher%27s_checksum * https://people.freebsd.org/~asomers/fletcher.pdf https://people.freebsd.org/~asomers/fletcher.pdf * https://www.intel.com/content/www/us/en/developer/articles/technical/fast-computation-of-fletcher-checksums.html https://www.intel.com/content/www/us/en/developer/articles/t... Originally documented in the paper "An Arithmetic Checksum for Serial Transmissions" (referenced also in RFC 1146, in the context of TCP).
- yjftsjthsd-h 11d ago
throw0101a 11d ago
> It's got CoW, snapshots, and data checksums. Therefore, it's good enough to compete with ZFS while being way smaller and permissively licensed. Does it have a built-in RAID layer? Because if it doesn't, then it can't compete with ZFS in many use cases. For example, what does "data may then be recovered from […] RAID restoration" mean? With ZFS, if you have a (e.g.) mirrored/RAID-1 configuration, and you fetch some data from one drive and the checksum is wrong, ZFS can check the other drive, and if that checksum is good it can (a) pass the good data up, and (b) use the good data to fix the bad data. Most mirroring systems can't do that both-drives checking: ZFS is self-healing. (This isn't to say that GEFS won't be useful in many other situations.)
- oridb 10d ago
  Author here: correct, it's currently on the user to deal with corrupted data. I'm not opposed to adding a RAID layer, but I also want to explore what adding trim-style feedback to a generic RAID layer may look like, so that different file systems could share the work. We'll see where things go.
  - throw0101a 10d ago
    > […] but I also want to explore what adding trim-style feedback to a generic RAID layer may look like […] TRIM commands sent to the block layer would probably help with SSD/NVMe wear leveling, as well as thin-provisioning in virtualized environments. And the lack of RAID is not a critique: if you're mostly interested in the file system layer that's fine, and a useful area to solve problems in. My comment was more towards the comparison to ZFS and its "rampant layering violation" [1] and some of the trade-offs that happen when you focus on more or fewer parts of the stack. [1] https://web.archive.org/web/20070602005153/http://blogs.sun.com/bonwick/entry/rampant_layering_violation https://web.archive.org/web/20070602005153/http://blogs.sun.... Unrelated: and since you're surfing the comments, with regards to your superblocks [1], you may wish to look into some of the lessons learned by ZFS; specifically see perhaps Allan Jude's "Large Label and Blockpointer-V2" from the 2025 OpenZFS Dev Summit: https://www.youtube.com/watch?v=3tqOBv8BmXI https://www.youtube.com/watch?v=3tqOBv8BmXI [1] https://orib.dev/gefs.pdf https://orib.dev/gefs.pdf § 9.1

atmosx 11d ago

I don't think it will compete with ZFS or BTRFS (e.g. I don't think ppl will use GEFS over ZFS or BTRFS for a storage server), but it's a modern, much needed FFS replacement.

yjftsjthsd-h 11d ago
Who said anything about storage servers? I'm using zfs on laptops and desktops right now because I want data checksums and a filesystem that doesn't have a history of breaking horribly (I dropped btrfs after the second time it hosed my rootfs). Given the license issue with zfs - and in particular, the technical fallout like needing dkms - I'd be very pleased to replace it.
- sippingabonedry 11d ago
  > I dropped btrfs after the second time it hosed my rootfs btrfs fans use the "you're using it wrong" excuse a lot. I recall a failure mode that activated when you fill the FS to 100% and their response was "you should never fill a filesystem to capacity"
  - jeffrallen 11d ago
    Otoh, good luck bringing a CoW filesystem back from 100%. Delete a file? Sure, let me just make a copy of all the metadata that was pointing at it using... the zero blocks I have left. Tradeoffs are a bitch, bitch.
    - yjftsjthsd-h 11d ago
      Which is why ZFS reserves "slop space" to make sure that doesn't happen, instead of defaulting to making it easy for users to corner themselves like that.
  - thetallguyyy 10d ago
    It's really a shame, though. btrfs is permanently unfinished, and zfs has the wrong license. End result is people stick with ext4 or xfs most of the time anyway.
  - scheme271 10d ago
    That failure mode is still there although it's been mitigated a bit. I hit it a few months ago but was able to recover after a few hours of doing various stuff.
    - sippingabonedry 10d ago
      Yeah that's not how filesystems are supposed to work. You should be able to dd /dev/zero to a file and not worry about the entire filesystem shitting the bed.
  - crest 10d ago
    They also fucked up their parity RAID repair so that the next disk failure on a different disk will destroy the data, argued that the pathological B-tree performance reported by a user couldn't be real, because they didn't understood the tree depth would only be logarithmic to the base of the fanout if the entries are the same size. insert shocked pikachu face
- whalesalad 11d ago
  I have been hearing noise recently that btrfs is risky and unstable but (knocks on wood) i've been running it for years now with zero issues. What am I missing?
  - crabbone 11d ago
    Failures in storage software are very rare. Which makes it very hard to test... (you need to run it a lot, for a very long time if you hope to find errors by chance). Also, some failure modes are worse than others. The failures known as DI (data integrity) are the worst. Even though they aren't expected to happen to everyone at a certain frequency (because, again, mature storage software is comparatively very reliable), even a single DI error that happened to any user sets up a major alarm. In the storage industry, the running joke is that after first DI in your product you lose funding, after the second DI you loose the product. And it did happen to Btrfs quite a bit... I've seen it with my own eyes when a system didn't come back after power failure. (But I'm in the business of testing software storage products, so, it's less surprising that it happened to me). So... it's perfectly plausible that you have never seen Btrfs fail, and it's been more error prone than eg. EXT4. The error rate is low enough so that if you don't actively try to cause the error you will never experience one. But, over a large group of diverse use patterns, the rate is still worse than expected.
    - hulitu 9d ago
      > Failures in storage software are very rare. Famous last words. Most failures are not reported, because people move on.
  - gucci-on-fleek 11d ago
    btrfs is almost 20 years old now, so lots of people only used it back when it was newer and far buggier. In my experience, btrfs is actually more reliable than other filesystems due to its checksumming abilities, but when it does fail, it's much harder to fix than with other filesystems (which will often try to continue on even when stuff is broken).
    - deleted 10d ago
- atmosx 11d ago
  > Who said anything about storage servers? I did. > I'm using zfs [...] ZFS is primarily used on single-storage appliances.
gigatexal 10d ago
It’ll replace brtfs if it gets ported. Almost anything is better than that pile of garbage.
- LargoLasskhyfv 10d ago
  Running it for more than two years now, without any hassles. On single SSDs though. But still, waddya even talkin about?
  - cestith 10d ago
    When I last took a serious look at btrfs its RAID 5 and RAID 6 modes could lose a whole lot of data if a single drive crashed or if the system lost power during a write. That's not supposed to happen on a RAID system with a journal. From what I understand single copy setups, RAID 1, and RAID 10 never had the same problem. It's left a bad taste for a lot of people though. Very early ext4 versions also had some data loss or data corruption, but I never hear people dismiss it completely for that brief spot of its history. Maybe part of that is because it was around 17 years ago and it's been a reliable default OS for many years now.
    - scheme271 10d ago
      The recommendation is still to store metadata on a raid 1 or 1/0 partition and the data on a raid 5/6. Apparently there's still some issues with raid 5/6
      - cestith 8d ago
        That's good information to have. Thanks.
  - throw0101a 9d ago
    > Running it for more than two years now, without any hassles. On single SSDs though. But still, waddya even talkin about? Running ZFS since it was originally released on Solaris (June 2006) with zero problems (then FreeBSD, then Linux), including on RAID-5/6 (RAID-Z1/2) configurations, which btrfs still does not recommend using all these years later: * https://btrfs.readthedocs.io/en/latest/btrfs-man5.html#raid56-status-and-recommended-practices https://btrfs.readthedocs.io/en/latest/btrfs-man5.html#raid5... What have the btrfs developer(s) been doing for the last decade(+) that the code is still of debatable dependability?

mmooss 11d ago

I've always wondered about similar designs: Doesn't calculating a hash of every block, on every read and every write, create lots of overhead? Why isn't that a problem? Some systems have dedicated crypto co-processors for confidentiality (encryption) - e.g., I think drives with FDE, and I think Apple Silicon SoCs might have them. Can those be repurposed for hash calculation? What about systems that lack them?

chasil 11d ago
Both ZFS and modern btrfs support a large set of checksums. Both implement sha256, which does impose a heavy speed penalty. ZFS allows you to adjust the checksum on the fly, using something faster (Fletcher) if desired. In btrfs, a global checksum is set at filesystem creation; xxhash is the best modern option. There is a website: https://xxhash.com https://xxhash.com Deduplication adds concerns for a strong hash free of collisions.
- ThePowerOfFuet 11d ago
  Fletcher has been the default for quite some time.
  - throw0101a 10d ago
    Specifically "fletcher4": * https://openzfs.github.io/openzfs-docs/man/master/7/zfsprops.7.html#checksum https://openzfs.github.io/openzfs-docs/man/master/7/zfsprops... * https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Data%20Storage/Checksums.html https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Data... * https://en.wikipedia.org/wiki/Fletcher%27s_checksum https://en.wikipedia.org/wiki/Fletcher%27s_checksum * https://people.freebsd.org/~asomers/fletcher.pdf https://people.freebsd.org/~asomers/fletcher.pdf * https://www.intel.com/content/www/us/en/developer/articles/technical/fast-computation-of-fletcher-checksums.html https://www.intel.com/content/www/us/en/developer/articles/t... Originally documented in the paper "An Arithmetic Checksum for Serial Transmissions" (referenced also in RFC 1146, in the context of TCP).
yjftsjthsd-h 11d ago

throw0101a 11d ago

> It's got CoW, snapshots, and data checksums. Therefore, it's good enough to compete with ZFS while being way smaller and permissively licensed. Does it have a built-in RAID layer? Because if it doesn't, then it can't compete with ZFS in many use cases. For example, what does "data may then be recovered from […] RAID restoration" mean? With ZFS, if you have a (e.g.) mirrored/RAID-1 configuration, and you fetch some data from one drive and the checksum is wrong, ZFS can check the other drive, and if that checksum is good it can (a) pass the good data up, and (b) use the good data to fix the bad data. Most mirroring systems can't do that both-drives checking: ZFS is self-healing. (This isn't to say that GEFS won't be useful in many other situations.)

oridb 10d ago
Author here: correct, it's currently on the user to deal with corrupted data. I'm not opposed to adding a RAID layer, but I also want to explore what adding trim-style feedback to a generic RAID layer may look like, so that different file systems could share the work. We'll see where things go.
- throw0101a 10d ago
  > […] but I also want to explore what adding trim-style feedback to a generic RAID layer may look like […] TRIM commands sent to the block layer would probably help with SSD/NVMe wear leveling, as well as thin-provisioning in virtualized environments. And the lack of RAID is not a critique: if you're mostly interested in the file system layer that's fine, and a useful area to solve problems in. My comment was more towards the comparison to ZFS and its "rampant layering violation" [1] and some of the trade-offs that happen when you focus on more or fewer parts of the stack. [1] https://web.archive.org/web/20070602005153/http://blogs.sun.com/bonwick/entry/rampant_layering_violation https://web.archive.org/web/20070602005153/http://blogs.sun.... Unrelated: and since you're surfing the comments, with regards to your superblocks [1], you may wish to look into some of the lessons learned by ZFS; specifically see perhaps Allan Jude's "Large Label and Blockpointer-V2" from the 2025 OpenZFS Dev Summit: https://www.youtube.com/watch?v=3tqOBv8BmXI https://www.youtube.com/watch?v=3tqOBv8BmXI [1] https://orib.dev/gefs.pdf https://orib.dev/gefs.pdf § 9.1