12 ms·

> zfs is technically great It's only great due to the lack of competitors in the checksummed-CoW-raid category. It lacks a bunch of things: Defrag, Reflinks, O

> zfs is technically great It's only great due to the lack of competitors in the checksummed-CoW-raid category. It lacks a bunch of things: Defrag, Reflinks, On-Demand Dedup, Rebalance (online raid geometry change, device removal, device shrink). It also wastes RAM due to page cache + ARC.

orra 3y ago
Sure, there's lots of room for improvement. IIRC, rebalancing might be a WIP, finally? But credit where credit is due: for a long time, ZFS has been the only fit for purpose filesystem, if you care about the integrity of your data.
- the8472 3y ago
  Afaik true rebalancing isn't in the works. Some limited add-device and remove-vdev features are in progress but AIUI they come with additional overhead and aren't as flexible. btrfs and bcachefs rebalance leave your pool as if you had created it from scratch with the existing data and the new layout.
e12e 3y ago
> [ZFS is] only great due to the lack of competitors in the checksummed-CoW-raid category. You forgot robust native encryption, network transparent dump/restore (ZFS send/receive) - and broad platform support (not so much anymore). For a while you could have a solid FS with encryption support for your USB hd that could be safely used with Linux, *BSD, Windows, Open/FOSS Solaris and MacOS.
- josephg 3y ago
  Is it just the implementation of zfs which is owned by oracle now? I wonder how hard it would be to write a compatible clean room reimplementation of zfs in rust or something, from the spec. Even if it doesn’t implement every feature from the real zfs, it would still be handy for OS compatibility reasons.
  - nine_k 3y ago
    I would suppose it would take years of effort? and a lot of testing in search of performance enhancements and elimination of corner cases. Even if the code of the FS itself is created in a provably correct manner (a very tall order even with Rust), real hardware has a lot of quirks which need to be addressed.
    - chasil 3y ago
      I wish the btrfs (and perhaps bcachefs) projects would collaborate with OpenZFS to rewrite equivalent code that they all used. It might take years, but washing Sun out of OpenZFS is the only thing that will free it.
      - mustache_kimono 3y ago
        OpenZFS is already free and open source. Linux kernel developers should just stop punching themselves in face. One way to solve the ZFS issue, Linus Torvalds could call a meeting of project leadership, and say, "Can we all agree that OpenZFS is not a derived work of Linux? It seems pretty obvious to anyone who understands the meaning of copyright term of art 'derived work' and the origin of ZFS ... Good. We shall add a commit which indicates such to the COPYING file [0], like we have for programs that interface at the syscall boundary to clear up any further confusion." Can you imagine trying to bring a copyright infringement suit (with no damages!) in such an instance? The ZFS hair shirt is a self imposed by semi-religious Linux wackadoos. [0]: See, https://github.com/torvalds/linux/blob/master/LICENSES/exceptions/Linux-syscall-note https://github.com/torvalds/linux/blob/master/LICENSES/excep...
        AshamedCaptain 3y ago
        Even if you were to be able to say that OpenZFS is not a derived work of Linux, all it would allow you to do is to distribute OpenZFS. You would _still_ not be able to distribute OpenZFS + Linux as a combined work. (I am one of these guys who thinks what Ubuntu is doing is crossing the line. To package two pieces of software whose license forbids you from distributing their combination in a way that "they are not combined but can be combined with a single click" is stretching it too much. ) It would be much simpler for Oracle to simply relicense older versions of ZFS under another license.
macdice 3y ago
Reflinks and copy_file_range() are just landing in OpenZFS now I think? (Block cloning)
- _piif 3y ago
  Block cloning support has indeed recently landed in git and already allows for reflinks under FreeBSD. Still has to be wired up for Linux though.
  - mustache_kimono 3y ago
    Really excited about this. Once support hits in Linux, a little app of mine[0] will support block cloning for its "roll forward" operation, where all previous snapshots are preserved, but a particular snapshot is rolled forward to the live dataset. Right now, data is simply diff copied in chunks. When this support hits, there will be no need to copy any data. Blocks written to the live dataset can just be references to the underlying snapshot blocks, and no extra space will need to be used. [0]: https://github.com/kimono-koans/httm https://github.com/kimono-koans/httm
    - nextaccountic 3y ago
      What does it mean to roll forward? I read the linked Github and I don't get what is happening > Roll forward to a previous ZFS snapshot, instead of rolling back (this avoids destroying interstitial snapshots): sudo httm --roll-forward=rpool/scratch@snap_2023-04-01-15:26:06_httmSnapFileMount [sudo] password for kimono: httm took a pre-execution snapshot named: rpool/scratch@snap_pre_2023-04-01-15:27:38_httmSnapRollForward ... httm roll forward completed successfully. httm took a post-execution snapshot named: rpool/scratch@snap_post_2023-04-01-15:28:40_:snap_2023-04-01-15:26:06_httmSnapFileMount:_httmSnapRollForward
      - mustache_kimono 3y ago
        From the help and man page[0]: --roll-forward="snap_name" traditionally 'zfs rollback' is a destructive operation, whereas httm roll-forward is non-destructive. httm will copy only files and their attributes that have changed since a specified snapshot, from that snapshot, to its live dataset. httm will also take two precautionary snapshots, one before and one after the copy. Should the roll forward fail for any reason, httm will roll back to the pre-execution state. Note: This is a ZFS only option which requires super user privileges. I might also add 'zfs rollback' is a destructive operation because it destroys snapshots between the current live version of the filesystem and the rollback snapshot target (the 'interstitial' snapshots). Imagine you have a ransom-ware installed and you need to rollback, but you want to view the ransomware's operations through snapshots for forensic purposes. You can do that. It's also faster than a checksummed rsync, because it makes a determination based on the underlying ZFS checksums, or more accurate than a non-checksummed rsync. This is a relatively minor feature re: httm. I recommend installing and playing around with it a bit. [0]: https://github.com/kimono-koans/httm/blob/master/httm.1 https://github.com/kimono-koans/httm/blob/master/httm.1
gigatexal 3y ago
“Wastes” ram? That’s a tunable my friend.
- quotemstr 3y ago
  Can I totally disable ARC yet?
  - vluft 3y ago
    I'm curious what your workflow is that not having any disk caching would have acceptable performance.
    - 0x457 3y ago
      A workflow where the person doesn't understand that RAM isn't wasted and it just their utility to show usage is wrong. Imagine being mad at file system cache being stored in RAM.
      - quotemstr 3y ago
        The problem with ARC in ZFS on Linux is the double caching. Linux already has a page cache. It doesn't need ZFS to provide a second page cache. I want to store things in the Linux page cache once, not once in the page cache and once in ZFS's special-sauce cache. If ARC is so good, it should be the general Linux page cache algorithm.
        curt15 3y ago
        >If ARC is so good, it should be the general Linux page cache algorithm Not possible until IBM's patent expires
        Dylan16807 3y ago
        Well patent 6,996,676 was filed in November 2002, which should mean it's expired now? I guess there's a few others listed in various places that were filed up to 2006-2008, I'm not sure how important they are.
ilyt 3y ago
Yeah world decided just replicating data somewhere is far preferable if you want to have resilience, instead of making the separate nodes more resilient.
mustache_kimono 3y ago
> It's only great due to the lack of competitors in the checksummed-CoW-raid category. blinks eyes, shakes head "It's only great because it's the only thing that's figured out how to do a hard thing really well" may be peak FOSS entitlement syndrome. Meanwhile, btrfs has rapidly gone nowhere, and, if you read the comments to this PR, bcachefs would love to get to simply nowhere/btrfs status, but is still years away. ZFS fulfills the core requirement of a filesystem, which is to store your data, such that when you read it back you can be assured it was the data you stored. It's amazing we continue to countenance systems that don't do this, simply because not fulfilling this core requirement was once considered acceptable.
- throw0101a 3y ago
  > Meanwhile, btrfs has rapidly gone nowhere […] A reminder that it came out in 2009: * https://en.wikipedia.org/wiki/Btrfs https://en.wikipedia.org/wiki/Btrfs (ext4 was declared stable in 2008.)
  - deepspace 3y ago
    Yes! File systems are hard. My prediction is that it will be *at least* 10 years before this newfangled FS gains both feature- and stability parity with BTRFS and ZFS. Also, BTRFS (albeit a modified version) has been used successfully in at least one commercial NAS (Synology), for many years. I don't see how that counts as "gone nowhere".
    - mustache_kimono 3y ago
      > Also, BTRFS (albeit a modified version) has been used successfully in at least one commercial NAS (Synology), for many years. I don't see how that counts as "gone nowhere". Excuse me for sounding glib. My point was btrfs isn't considered a serious competitor to ZFS in many of the spaces ZFS operates. Moreover, it's inability to do RAID5/6 after years of effort is just weird now.
      - PlutoIsAPlanet 3y ago

orra 3y ago

Sure, there's lots of room for improvement. IIRC, rebalancing might be a WIP, finally? But credit where credit is due: for a long time, ZFS has been the only fit for purpose filesystem, if you care about the integrity of your data.

the8472 3y ago
Afaik true rebalancing isn't in the works. Some limited add-device and remove-vdev features are in progress but AIUI they come with additional overhead and aren't as flexible. btrfs and bcachefs rebalance leave your pool as if you had created it from scratch with the existing data and the new layout.

e12e 3y ago

> [ZFS is] only great due to the lack of competitors in the checksummed-CoW-raid category. You forgot robust native encryption, network transparent dump/restore (ZFS send/receive) - and broad platform support (not so much anymore). For a while you could have a solid FS with encryption support for your USB hd that could be safely used with Linux, *BSD, Windows, Open/FOSS Solaris and MacOS.

josephg 3y ago
Is it just the implementation of zfs which is owned by oracle now? I wonder how hard it would be to write a compatible clean room reimplementation of zfs in rust or something, from the spec. Even if it doesn’t implement every feature from the real zfs, it would still be handy for OS compatibility reasons.
- nine_k 3y ago
  I would suppose it would take years of effort? and a lot of testing in search of performance enhancements and elimination of corner cases. Even if the code of the FS itself is created in a provably correct manner (a very tall order even with Rust), real hardware has a lot of quirks which need to be addressed.
  - chasil 3y ago
    I wish the btrfs (and perhaps bcachefs) projects would collaborate with OpenZFS to rewrite equivalent code that they all used. It might take years, but washing Sun out of OpenZFS is the only thing that will free it.
    - mustache_kimono 3y ago
      OpenZFS is already free and open source. Linux kernel developers should just stop punching themselves in face. One way to solve the ZFS issue, Linus Torvalds could call a meeting of project leadership, and say, "Can we all agree that OpenZFS is not a derived work of Linux? It seems pretty obvious to anyone who understands the meaning of copyright term of art 'derived work' and the origin of ZFS ... Good. We shall add a commit which indicates such to the COPYING file [0], like we have for programs that interface at the syscall boundary to clear up any further confusion." Can you imagine trying to bring a copyright infringement suit (with no damages!) in such an instance? The ZFS hair shirt is a self imposed by semi-religious Linux wackadoos. [0]: See, https://github.com/torvalds/linux/blob/master/LICENSES/exceptions/Linux-syscall-note https://github.com/torvalds/linux/blob/master/LICENSES/excep...
      - AshamedCaptain 3y ago
        Even if you were to be able to say that OpenZFS is not a derived work of Linux, all it would allow you to do is to distribute OpenZFS. You would _still_ not be able to distribute OpenZFS + Linux as a combined work. (I am one of these guys who thinks what Ubuntu is doing is crossing the line. To package two pieces of software whose license forbids you from distributing their combination in a way that "they are not combined but can be combined with a single click" is stretching it too much. ) It would be much simpler for Oracle to simply relicense older versions of ZFS under another license.

macdice 3y ago

Reflinks and copy_file_range() are just landing in OpenZFS now I think? (Block cloning)

_piif 3y ago
Block cloning support has indeed recently landed in git and already allows for reflinks under FreeBSD. Still has to be wired up for Linux though.
- mustache_kimono 3y ago
  Really excited about this. Once support hits in Linux, a little app of mine[0] will support block cloning for its "roll forward" operation, where all previous snapshots are preserved, but a particular snapshot is rolled forward to the live dataset. Right now, data is simply diff copied in chunks. When this support hits, there will be no need to copy any data. Blocks written to the live dataset can just be references to the underlying snapshot blocks, and no extra space will need to be used. [0]: https://github.com/kimono-koans/httm https://github.com/kimono-koans/httm
  - nextaccountic 3y ago
    What does it mean to roll forward? I read the linked Github and I don't get what is happening > Roll forward to a previous ZFS snapshot, instead of rolling back (this avoids destroying interstitial snapshots): sudo httm --roll-forward=rpool/scratch@snap_2023-04-01-15:26:06_httmSnapFileMount [sudo] password for kimono: httm took a pre-execution snapshot named: rpool/scratch@snap_pre_2023-04-01-15:27:38_httmSnapRollForward ... httm roll forward completed successfully. httm took a post-execution snapshot named: rpool/scratch@snap_post_2023-04-01-15:28:40_:snap_2023-04-01-15:26:06_httmSnapFileMount:_httmSnapRollForward
    - mustache_kimono 3y ago
      From the help and man page[0]: --roll-forward="snap_name" traditionally 'zfs rollback' is a destructive operation, whereas httm roll-forward is non-destructive. httm will copy only files and their attributes that have changed since a specified snapshot, from that snapshot, to its live dataset. httm will also take two precautionary snapshots, one before and one after the copy. Should the roll forward fail for any reason, httm will roll back to the pre-execution state. Note: This is a ZFS only option which requires super user privileges. I might also add 'zfs rollback' is a destructive operation because it destroys snapshots between the current live version of the filesystem and the rollback snapshot target (the 'interstitial' snapshots). Imagine you have a ransom-ware installed and you need to rollback, but you want to view the ransomware's operations through snapshots for forensic purposes. You can do that. It's also faster than a checksummed rsync, because it makes a determination based on the underlying ZFS checksums, or more accurate than a non-checksummed rsync. This is a relatively minor feature re: httm. I recommend installing and playing around with it a bit. [0]: https://github.com/kimono-koans/httm/blob/master/httm.1 https://github.com/kimono-koans/httm/blob/master/httm.1

gigatexal 3y ago

“Wastes” ram? That’s a tunable my friend.

quotemstr 3y ago
Can I totally disable ARC yet?
- vluft 3y ago
  I'm curious what your workflow is that not having any disk caching would have acceptable performance.
  - 0x457 3y ago
    A workflow where the person doesn't understand that RAM isn't wasted and it just their utility to show usage is wrong. Imagine being mad at file system cache being stored in RAM.
    - quotemstr 3y ago
      The problem with ARC in ZFS on Linux is the double caching. Linux already has a page cache. It doesn't need ZFS to provide a second page cache. I want to store things in the Linux page cache once, not once in the page cache and once in ZFS's special-sauce cache. If ARC is so good, it should be the general Linux page cache algorithm.
      - curt15 3y ago
        >If ARC is so good, it should be the general Linux page cache algorithm Not possible until IBM's patent expires
        Dylan16807 3y ago
        Well patent 6,996,676 was filed in November 2002, which should mean it's expired now? I guess there's a few others listed in various places that were filed up to 2006-2008, I'm not sure how important they are.

ilyt 3y ago

Yeah world decided just replicating data somewhere is far preferable if you want to have resilience, instead of making the separate nodes more resilient.

mustache_kimono 3y ago

> It's only great due to the lack of competitors in the checksummed-CoW-raid category. blinks eyes, shakes head "It's only great because it's the only thing that's figured out how to do a hard thing really well" may be peak FOSS entitlement syndrome. Meanwhile, btrfs has rapidly gone nowhere, and, if you read the comments to this PR, bcachefs would love to get to simply nowhere/btrfs status, but is still years away. ZFS fulfills the core requirement of a filesystem, which is to store your data, such that when you read it back you can be assured it was the data you stored. It's amazing we continue to countenance systems that don't do this, simply because not fulfilling this core requirement was once considered acceptable.

throw0101a 3y ago
> Meanwhile, btrfs has rapidly gone nowhere […] A reminder that it came out in 2009: * https://en.wikipedia.org/wiki/Btrfs https://en.wikipedia.org/wiki/Btrfs (ext4 was declared stable in 2008.)
- deepspace 3y ago
  Yes! File systems are hard. My prediction is that it will be *at least* 10 years before this newfangled FS gains both feature- and stability parity with BTRFS and ZFS. Also, BTRFS (albeit a modified version) has been used successfully in at least one commercial NAS (Synology), for many years. I don't see how that counts as "gone nowhere".
  - mustache_kimono 3y ago
    > Also, BTRFS (albeit a modified version) has been used successfully in at least one commercial NAS (Synology), for many years. I don't see how that counts as "gone nowhere". Excuse me for sounding glib. My point was btrfs isn't considered a serious competitor to ZFS in many of the spaces ZFS operates. Moreover, it's inability to do RAID5/6 after years of effort is just weird now.
    - PlutoIsAPlanet 3y ago