3 ms·
> You really can't blame ZFS for needing ram for dedup. The raw math around looking up checksum just demands it. There are some other systems that can achieve f
by LeoPanthera 8y ago
> You really can't blame ZFS for needing ram for dedup. The raw math around looking up checksum just demands it. There are some other systems that can achieve fast dedup, but instead of ram they would require a ton of super fast SSDs to store the tables.
This is only true for real-time, "live" dedup, which is an essential feature for almost nothing.
btrfs' offline dedup (which, despite the name, can be done on a mounted filesystem) is far more rational. You fill the disk with data, and then dedup it later, at a schedule that works for you. No extra RAM needed.
- mbrumlow 8y ago> btrfs' offline dedup (which, despite the name, can be done on a mounted filesystem) is far more rational. You still have to process it, and you are still going to use a lot of ram -- and you are going to want to use a lot of ram once you get into larger data sets otherwise the process will take days.