4 ms·
If writing performance is critical, why bother with deduplication at writing time? Do deduplication afterwards, concurrently and with lower priority?
by tilt_error 2y ago
If writing performance is critical, why bother with deduplication at writing time? Do deduplication afterwards, concurrently and with lower priority?
- klysm 2y agoKinda like log structured merge tree?
- 0x457 2y agoBecause to make this work without a lot of copying, you would need to mutate things that ZFS absolutely does not want to make mutable.
- UltraSane 2y agoIf the block to be written is already being stored then you will match the hash and the block won't have to be written. This can save a lot of write IO in real world use.
- magicalhippo 2y agoKeep in mind ZFS was created at a time when disks were glacial in comparison to CPUs. And, the fastest write is the one you don't perform, so you can afford some CPU time to check for duplicate blocks. That said, NVMe has changed that balance a lot, and you can afford a lot less before you're bottlenecking the drives.