6 ms·
zstd is faster and provides better compression than gzip at every point on the curve. There is no reason to use gzip these days other than backwards compatibil
by loeg 2y ago
zstd is faster and provides better compression than gzip at every point on the curve. There is no reason to use gzip these days other than backwards compatibility.
- idoubtit 2y agoI've reached the same conclusion, and I've been using zstd unless I want extremely fast (de)compression with lz4. And even when I still have to use gzip (the format), the executables of libdeflate are fully compatible and noticeably faster than gzip/gunzip (the tool).
- jrockway 2y agoThis was on HN a while ago https://jolynch.github.io/posts/use_fast_data_algorithms/ https://jolynch.github.io/posts/use_fast_data_algorithms/ and reaches the same conclusion. I am inclined to use Zstd even in cases where people would use lz4. Zstd is very fast.
- ThatGuyRaion 2y agozstd is a great setup, for sure, but the build system they use is patently awful. Can someone make an autoconf or hell, even CMake build system for them pleasee???
- loeg 2y agoJust install the binary package from whatever distro you use? Why do you need to build it? But if it matters, FreeBSD has a pretty trivial BSDmake build of it: https://github.com/freebsd/freebsd-src/blob/main/lib/libzstd/Makefile https://github.com/freebsd/freebsd-src/blob/main/lib/libzstd... https://github.com/freebsd/freebsd-src/blob/main/usr.bin/zstd/Makefile https://github.com/freebsd/freebsd-src/blob/main/usr.bin/zst... You could easily do something similar in GNU make or whatever without the dependencies on the FBSD build system. It's basically just the classic "cc -c a bunch of C files, then link them."
- jrockway 2y agoI suppose the point of open source software is to edit the source code and make your own version. If you can't build it, you can't do that.
- loeg 2y agoZstd has lots of contributors who have all figured out how to edit and build it.
- ThatGuyRaion 2y agoIt's basically a giant GCCism. Don't be condescending.
- loeg 2y agoI have no idea what you're talking about at this point.
- jrockway 2y agoYeah, not sure what the original commenter is complaining about. I did "git clone https://github.com/facebook/zstd https://github.com/facebook/zstd", "cd zstd", "make" and have a working binary. Doesn't get easier than that.
- ThatGuyRaion 2y agoNot everyone is using a standard tool chain or anything like that and oftentimes when I find projects with atypical build systems that give me trouble I tend to just move on to something else
- ThatGuyRaion 2y ago> Just install the binary package from whatever distro you use? Why do you need to build it? Cuz I'm actually a packaging maintainer for a couple different old operating systems. Regardless I thank you for being thankful even though I feel like you were being backhanded and demeaning on a number of levels.
- MawKKe 2y agoQuick glance of zstd github repo shows they do provide CMake build scripts?
- ThatGuyRaion 2y agoLast I touched it they only had glitchy make files and I don't like cmake particularly, I was just like "bespoke make files are fine if they're posix compliant, but these aren't"
- hinkley 2y agoDo zless, zcat and zgrep support zstd everywhere? And I mean everywhere? VMs? Alpine? FreeBSD? OSX? Openwrt? Nothing is shittier than sshing into a box that doesn’t understand half of your command line tricks. Or the clever shell script you just tested six ways to Sunday. It’s like fighting with your hands tied behind your back.
- sophiebits 2y ago> other than backwards compatibility
- hinkley 2y agoOther than that, Mrs Lincoln, how was the play?
- adastra22 2y agoYes, it's supported on all those platforms.
- skrebbel 2y ago> There is no reason to use gzip these days other than backwards compatibility And forward compatibility. The Lindy effect says gzip is likelier to be widely available across platforms and tools in the long term than zstd.
- t43562 2y ago> bzip2 -9 -k all.tar 981.78s user 9.77s system 95% cpu 8M memory 17:16.64 total > bzip3 -e -b 256 -j 12 all.tar 2713.81s user 16.28s system 634% cpu 18301M memory 7:10.10 total The memory usage is one reason: 8M vs 18301M