3 ms·

Zstd fully supports binary deltas. If you're computing a delta between `old` and `new` you would do this to round trip it: zstd --patch-from=old new -c | zstd

Zstd fully supports binary deltas. If you're computing a delta between `old` and `new` you would do this to round trip it: zstd --patch-from=old new -c | zstd -d --patch-from=old Using the library this can be achieved by using the old file as a dictionary. If you have very large files (MB-GB range) you will also want to enable long range mode, otherwise zstd might not be able to find the repetition. --patch-from on the CLI will manage this automatically. The release notes from zstd-1.4.5 [0] contain some details about the speed/size tradeoff of "patch-from" mode, compared to other delta engines. There is also a wiki page detailing how to use zstd as a delta engine [1]. [0] https://github.com/facebook/zstd/releases/tag/v1.4.5 https://github.com/facebook/zstd/releases/tag/v1.4.5 [1] https://github.com/facebook/zstd/wiki/Zstandard-as-a-patching-engine https://github.com/facebook/zstd/wiki/Zstandard-as-a-patchin...

mips_avatar 6y ago
I don't think you are doing the correct comparison in your article. BSDiff is two steps, the first step is calculating the binary subtraction mappings, and then compressing these mappings with a compressor. Colin Percival used zlib compression in his original bsdiff paper, but the performance of zlib is not great. If you really wanted to compare zstd deltas with bsdiff you need to replace zlib with a modern compressor like regular old zstd.
- terrelln 6y ago
  You're right that a totally fair comparison would use zstd. I didn't personally run these benchmarks, but I believe that the dominate time was computing the binary subtraction mappings, so I would expect similar results.
- terrelln 6y ago
  Also, if you give zstd binary delta a try, we would be super interested in hearing the results you get! Especially around the tradeoff between compressed size, compression time, and decompression time. Opening an issue would be the best way to communicate it.

mips_avatar 6y ago

I don't think you are doing the correct comparison in your article. BSDiff is two steps, the first step is calculating the binary subtraction mappings, and then compressing these mappings with a compressor. Colin Percival used zlib compression in his original bsdiff paper, but the performance of zlib is not great. If you really wanted to compare zstd deltas with bsdiff you need to replace zlib with a modern compressor like regular old zstd.

terrelln 6y ago
You're right that a totally fair comparison would use zstd. I didn't personally run these benchmarks, but I believe that the dominate time was computing the binary subtraction mappings, so I would expect similar results.
terrelln 6y ago
Also, if you give zstd binary delta a try, we would be super interested in hearing the results you get! Especially around the tradeoff between compressed size, compression time, and decompression time. Opening an issue would be the best way to communicate it.