4 ms·
VMs are known to benefit from dedupe so yes, you'll see benefits there. ZFS is a general-purpose filesystem not just an enterprise SAN so many ZFS users aren't
by wmf 2y ago
VMs are known to benefit from dedupe so yes, you'll see benefits there. ZFS is a general-purpose filesystem not just an enterprise SAN so many ZFS users aren't running VMs.
Dedupe/compression works really well on syslog
I apologize for the pedantry but dedupe and compression aren't the same thing (although they tend to be bundled in the enterprise storage world). Logs are probably benefiting from compression not dedupe and ZFS had compression all along.
- ants_everywhere 2y agoIf we're being pedants, then storing the same information in fewer bits than the input is by definition a form of compression, no? (Although yes I understand that file-level compression with a standard algorithm is a different thing than dedup)
- tw04 2y agoThey are not the same thing, but when you boil it down to the raw math, they aren't identical twins, but they're absolutely fraternal twins. Both are trying to eliminate repeating data, it's just the frame of reference that changes. Compression in this context is operating on a given block or handful of blocks. Deduplication is operating on the entire "volume" of data. "Volume" having a different meaning depending on the filesystem/storage array in question.
- UltraSane 2y agoWell put. I like to say compression is just short range dedupe. Hash based dedupe wouldn't be needed if you could just to real-time LZMA on all of the data on a storage array but that just isn't feasible and hash-based dedupe is a very effective compromise.
- ShroudedNight 2y agoIs "paternal twins" a linguistic borrowing of some sort? It seems a relatively novel form of what I've mostly seen referred to as monozygotic / 'identical' twins. Searching for some kind of semi-canonical confirmation of its widespread use turns up one, maybe two articles where it's treated as an orthodox term, and at least an equal number of discussions admonishing its use.
- spockz 2y agoIf anything I would expect the term “maternal” twin to be used as whether or not a twin is monozygotic or “identical” depends on the amount of eggs from the mother.
- xxs 2y agocompression tends NOT to use a global dictionary. So to me they are vastly different even if they have the same goal of reducing the output size. Compression with a global dict would like do better than dedup yet it will have a lot of other issues.