4 ms·
A curious fact I encountered: First, the zstdcat executable that ships with zstd (actually a symlink to zstd itself, roughly equivalent to "zstd -cqdf") can dec
by waterhouse 9y ago
A curious fact I encountered: First, the zstdcat executable that ships with zstd (actually a symlink to zstd itself, roughly equivalent to "zstd -cqdf") can decompress .gz files. Second, zstd is faster than gzip at decompressing .gz files, taking about 2/3 as long.
I'm not kidding. I couldn't believe it myself, but subsequent testing stubbornly bore it out—on one file that was 15 MB after compression, and on a mix of smaller files. I tried compiling gzip from source, using the same compiler I used for zstd, and the results were the same. strace seemed to show zstd read and wrote in chunks 2x the size, but the number of syscalls didn't seem to be nearly enough to explain the difference. It seems to have this "zlibWrapper" subdirectory; its README has some benchmark numbers that I don't fully understand, but some of them seem to match the 2/3 factor I got.
I'm wondering if this is a clever tactic to drive adoption—get people to use the zstd executable even when they're still using gzipped files. ;-)
Also, the fact that it has support for reading (and, apparently, writing) gzip, lz4, and xz on top of its own format really makes "z standard" an appropriate name.
- emj 9y agoA zlib feature is being patent free, that might very well affect speed. I do not know if that is the issue you are describing.
- waterhouse 9y agoSo, the zstd executable links to libz.so (says ldd), and I don't think it reimplements the core algorithmic part of gzip. I gather that it's simply more intelligent in how it uses the API... e.g. it says some things about reusing contexts, though that seems to apply only to handling multiple files.