3 ms·
Have seen zstd pop up frequently in zfs and hammer2 discussions, have been running lz4 on FS (and in some cases in for RAM-compressions) for a while. From my p
by hultner 8y ago
Have seen zstd pop up frequently in zfs and hammer2 discussions, have been running lz4 on FS (and in some cases in for RAM-compressions) for a while.
From my point of view zstd looks like a very interesting alternative to gzip since it's an order of magnitude faster in tests I've seen.
But lz4 seems to still be the champion for raw throughput speed with decent compression, this might change (have changed?) with the negative compression modes in zstd.
It would be interesting to hear from people who've got a bit more hands on experience with zstd in theses contexts.
The dictionary training, would that be applicable on a dataset/volume in a FS context? It would be awesome if for instance I have a dataset for jpg and another for raw-photos and I could get some good compressions for those.
Media usually yields quite bad compression ratios using more traditional compression formats, dedupe can improve this some but usually requires large DDTs (deduplication tables). Could the dictionary training be an alternative in these cases?