5 ms·

zfs is really a study in cli ergonomics. It has a quite idiosyncratic CLI, which one might assume is a bad thing. But the CLI is so well thought out that learni

zfs is really a study in cli ergonomics. It has a quite idiosyncratic CLI, which one might assume is a bad thing. But the CLI is so well thought out that learning this new "language" is quite intuitive. btrfs uses a more traditional command, subcommand, args setup, which by itself is not a problem, but the actual command and subcommand structure is a dumpster fire to be charitable. Try managing large groups of snapshots without custom tooling. The whole thing makes me sad really. btrfs has a lot going for it at the architecture and disk format level (dedup is actually something theoretically useful, while the zfs design was flawed from the start), but the implementation has just never made it.

rkagerer 7y ago
Dumb question as I haven't used the feature: Why is dedupe flawed? Is it because it requires "enormous" amounts of RAM? Does it eventually slow down writes?
- braindeath 7y ago
  It's not just the issue of RAM, but that dedupe is only attainable with the online DDT method. There is no offline deduping at the block level, and of course related annoying things like no COW between filesystems.
- rincebrain 7y ago
  Dedup on ZFS is problematic because ZFS, in exchange for some of its core useful features, promises that block locations on disk are immutable. So the only place you can do dedup is inline, as the data is being written the first time, not after-the-fact. In addition, this requires you keep a huge indirection table (the DDT, or dedup table) that needs to be read for all writes, so either that has to be kept in memory or on fast storage, or you've just turned every write into one or more random reads, plus writes. This also means that even if you turn off dedup after turning it on, the performance implications remain until the DDT no longer contains any blocks (e.g. you rewrote all the data after turning dedup off). There are feature proposals to make the performance of dedup less pathological, but nobody's taken up implementing them so far. (Someone even did a proof of concept implementation of one of them, and it still hasn't been finished and integrated.)
- 693471 7y ago
  Dedup was rushed and the original developers admit it. They ask you to please not use it unless you absolutely know what you're doing
jannotti 7y ago
Could you say more about each, with a couple examples? I find myself building little CLIs all the time, yet haven't developed any real intuition about what makes a good one.
- pletnes 7y ago
  Follow conventions. Python’s click library is great in this respect, reading its docs is useful. Having —help and —version flags; -s and —long options (one dash for short, two for long); etc.

rkagerer 7y ago

Dumb question as I haven't used the feature: Why is dedupe flawed? Is it because it requires "enormous" amounts of RAM? Does it eventually slow down writes?

braindeath 7y ago
It's not just the issue of RAM, but that dedupe is only attainable with the online DDT method. There is no offline deduping at the block level, and of course related annoying things like no COW between filesystems.
rincebrain 7y ago
Dedup on ZFS is problematic because ZFS, in exchange for some of its core useful features, promises that block locations on disk are immutable. So the only place you can do dedup is inline, as the data is being written the first time, not after-the-fact. In addition, this requires you keep a huge indirection table (the DDT, or dedup table) that needs to be read for all writes, so either that has to be kept in memory or on fast storage, or you've just turned every write into one or more random reads, plus writes. This also means that even if you turn off dedup after turning it on, the performance implications remain until the DDT no longer contains any blocks (e.g. you rewrote all the data after turning dedup off). There are feature proposals to make the performance of dedup less pathological, but nobody's taken up implementing them so far. (Someone even did a proof of concept implementation of one of them, and it still hasn't been finished and integrated.)
693471 7y ago
Dedup was rushed and the original developers admit it. They ask you to please not use it unless you absolutely know what you're doing

jannotti 7y ago

Could you say more about each, with a couple examples? I find myself building little CLIs all the time, yet haven't developed any real intuition about what makes a good one.

pletnes 7y ago
Follow conventions. Python’s click library is great in this respect, reading its docs is useful. Having —help and —version flags; -s and —long options (one dash for short, two for long); etc.