4 ms·

I've been super happy with ZFS for over a decade. All the pain I've had with it has been related to deduplication requiring massive amounts of RAM. When you d

I've been super happy with ZFS for over a decade. All the pain I've had with it has been related to deduplication requiring massive amounts of RAM. When you don't have the RAM, it tends to be flaky. But I've never run into data loss with it, despite my best attempts. My current backup server would need ~50GB of RAM just for the in-RAM deduplication tables. I've been thinking of trying btrfs for the backup server, but the deduplication story is not something that's obvious, it all seems to be third-party, which makes me skeptical. But it worth trying. I might just try a test Dragonfly setup, because my simple testing of HAMMER seemed very promising, years ago. I've had pretty bad experiences with btrfs in the distant past. Around a decade ago I had all my company laptops using btrfs and it worked great, as long as we didn't get >80% usage or something. It worked well for around a year. But then the reliability of btrfs dropped, and we had repeated data loss and switched back to ext.

poizan42 8y ago
> I've had pretty bad experiences with btrfs in the distant past. Around a decade ago I had all my company laptops using btrfs and it worked great, as long as we didn't get >80% usage or something. It worked well for around a year. But then the reliability of btrfs dropped, and we had repeated data loss and switched back to ext. btrfs wasn't considered stable until 2012. Using an experimental filesystem for all your company laptops seems to be asking for trouble.
mbrumlow 8y ago
You really can't blame ZFS for needing ram for dedup. The raw math around looking up checksum just demands it. There are some other systems that can achieve fast dedup, but instead of ram they would require a ton of super fast SSDs to store the tables. There are some other types of dedup that don't guarantee 100% dedup, but then the dedup levels would very depending on data sets -- this may be fine for VMs and user data. They work by being using data locality, predicting which dedup table would needed to dedup the next N bytes. But it is not likely we are going to see this any time soon because the patents on this sort of dedup is fairly recent.
- LeoPanthera 8y ago
  > You really can't blame ZFS for needing ram for dedup. The raw math around looking up checksum just demands it. There are some other systems that can achieve fast dedup, but instead of ram they would require a ton of super fast SSDs to store the tables. This is only true for real-time, "live" dedup, which is an essential feature for almost nothing. btrfs' offline dedup (which, despite the name, can be done on a mounted filesystem) is far more rational. You fill the disk with data, and then dedup it later, at a schedule that works for you. No extra RAM needed.
  - mbrumlow 8y ago
    > btrfs' offline dedup (which, despite the name, can be done on a mounted filesystem) is far more rational. You still have to process it, and you are still going to use a lot of ram -- and you are going to want to use a lot of ram once you get into larger data sets otherwise the process will take days.
- tatersolid 8y ago
  Most sane deduplication systems use a Bloom filter to avoid direct lookups to huge tables of content hashes. Not sure if ZFS does this or not, but even NTFS deduplication does.

poizan42 8y ago

> I've had pretty bad experiences with btrfs in the distant past. Around a decade ago I had all my company laptops using btrfs and it worked great, as long as we didn't get >80% usage or something. It worked well for around a year. But then the reliability of btrfs dropped, and we had repeated data loss and switched back to ext. btrfs wasn't considered stable until 2012. Using an experimental filesystem for all your company laptops seems to be asking for trouble.

mbrumlow 8y ago

You really can't blame ZFS for needing ram for dedup. The raw math around looking up checksum just demands it. There are some other systems that can achieve fast dedup, but instead of ram they would require a ton of super fast SSDs to store the tables. There are some other types of dedup that don't guarantee 100% dedup, but then the dedup levels would very depending on data sets -- this may be fine for VMs and user data. They work by being using data locality, predicting which dedup table would needed to dedup the next N bytes. But it is not likely we are going to see this any time soon because the patents on this sort of dedup is fairly recent.

LeoPanthera 8y ago
> You really can't blame ZFS for needing ram for dedup. The raw math around looking up checksum just demands it. There are some other systems that can achieve fast dedup, but instead of ram they would require a ton of super fast SSDs to store the tables. This is only true for real-time, "live" dedup, which is an essential feature for almost nothing. btrfs' offline dedup (which, despite the name, can be done on a mounted filesystem) is far more rational. You fill the disk with data, and then dedup it later, at a schedule that works for you. No extra RAM needed.
- mbrumlow 8y ago
  > btrfs' offline dedup (which, despite the name, can be done on a mounted filesystem) is far more rational. You still have to process it, and you are still going to use a lot of ram -- and you are going to want to use a lot of ram once you get into larger data sets otherwise the process will take days.
tatersolid 8y ago
Most sane deduplication systems use a Bloom filter to avoid direct lookups to huge tables of content hashes. Not sure if ZFS does this or not, but even NTFS deduplication does.