3 ms·

Three months ago I converted an old Core 2 Duo desktop machine to a server and also deliberated whether to use btrfs or ZFS. What put me off btrfs were the roug

by mrich 6y ago

Three months ago I converted an old Core 2 Duo desktop machine to a server and also deliberated whether to use btrfs or ZFS. What put me off btrfs were the rough edges, e.g. some RAID setups are considered beta. In ZFS those have been stable for a long time. The fact that btrfs has been in development for quite a long time and still has such issues is not very assuring to me. That said, if you only use the features big orgs are using, you are probably fine. Performance has been great so far even on my underpowered machine, even with just 4GB. I don't use deduplication though which makes a huge difference. I chose Ubuntu 20.04 as OS since they are pushing ZFS support. Did consider FreeBSD as well where I had a positive experience in the past but since they are switching to OpenZFS anyway I stuck with Ubuntu since I'm running Debian derivatives on all my servers.

hardwaresofton 6y ago
Thanks for sharing this insight! > Performance has been great so far even on my underpowered machine, even with just 4GB. I don't use deduplication though which makes a huge difference. So almost every OpenZFS community video/talk I've seen recently has included a like like "friends don't let friends dedup" or something to that effect... I think dedup is considered unnecessary/dangerous these days with how good compression is. Not sure exactly what the dangers were, but I know that I wouldn't even turn it on. > I chose Ubuntu 20.04 as OS since they are pushing ZFS support. Did consider FreeBSD as well where I had a positive experience in the past but since they are switching to OpenZFS anyway I stuck with Ubuntu since I'm running Debian derivatives on all my servers. Same, I used to run lots of different OSes but I've settled down on Ubuntu for everything now, and OpenZFS having good support is what makes it possible there.
- magicalhippo 6y ago
  > Not sure exactly what the dangers were No expert either but from what I've gathered the main issue is that the dedup tables require a _lot_ of memory that scales with the size of the pool, and if they can't fit in RAM performance tanks. However the real issue is that any blocks written while dedup was enabled will demand this dedup overhead, even if the dedup option is turned off. That is, it's a "sticky feature". So if you use dedup, notice performance tanks because dedup tables are too big, well you're screwed because turning it off won't give you back the performance. Only way to recover is to send/receive the whole shebang to a separate pool. In addition, they don't actually bring that much space saving on common workloads. Most people don't have a lot of truly identical blocks of data.
  - benlivengood 6y ago
    > In addition, they don't actually bring that much space saving on common workloads. Most people don't have a lot of truly identical blocks of data. VM backing storage seems to be the biggest worthwhile use case, but that depends on whether snapshots and clones are used extensively. Installing 100 copies of Debian on empty VMs will likely get deduped quite a bit. But it's faster and provides almost the same benefits to install one VM, snapshot it, and produce the rest of the VMs by cloning from the snapshot. The only other case I could imagine dedup being good for is storing a lot of genomic data: https://techtransfer.universityofcalifornia.edu/NCD/25080.html https://techtransfer.universityofcalifornia.edu/NCD/25080.ht... But if the use-case is narrow enough for custom deduplication then it will probably be much more efficient than ZFS's block-based dedup.

by mrich 6y ago

hardwaresofton 6y ago

Thanks for sharing this insight! > Performance has been great so far even on my underpowered machine, even with just 4GB. I don't use deduplication though which makes a huge difference. So almost every OpenZFS community video/talk I've seen recently has included a like like "friends don't let friends dedup" or something to that effect... I think dedup is considered unnecessary/dangerous these days with how good compression is. Not sure exactly what the dangers were, but I know that I wouldn't even turn it on. > I chose Ubuntu 20.04 as OS since they are pushing ZFS support. Did consider FreeBSD as well where I had a positive experience in the past but since they are switching to OpenZFS anyway I stuck with Ubuntu since I'm running Debian derivatives on all my servers. Same, I used to run lots of different OSes but I've settled down on Ubuntu for everything now, and OpenZFS having good support is what makes it possible there.

magicalhippo 6y ago
> Not sure exactly what the dangers were No expert either but from what I've gathered the main issue is that the dedup tables require a _lot_ of memory that scales with the size of the pool, and if they can't fit in RAM performance tanks. However the real issue is that any blocks written while dedup was enabled will demand this dedup overhead, even if the dedup option is turned off. That is, it's a "sticky feature". So if you use dedup, notice performance tanks because dedup tables are too big, well you're screwed because turning it off won't give you back the performance. Only way to recover is to send/receive the whole shebang to a separate pool. In addition, they don't actually bring that much space saving on common workloads. Most people don't have a lot of truly identical blocks of data.
- benlivengood 6y ago
  > In addition, they don't actually bring that much space saving on common workloads. Most people don't have a lot of truly identical blocks of data. VM backing storage seems to be the biggest worthwhile use case, but that depends on whether snapshots and clones are used extensively. Installing 100 copies of Debian on empty VMs will likely get deduped quite a bit. But it's faster and provides almost the same benefits to install one VM, snapshot it, and produce the rest of the VMs by cloning from the snapshot. The only other case I could imagine dedup being good for is storing a lot of genomic data: https://techtransfer.universityofcalifornia.edu/NCD/25080.html https://techtransfer.universityofcalifornia.edu/NCD/25080.ht... But if the use-case is narrow enough for custom deduplication then it will probably be much more efficient than ZFS's block-based dedup.