3 ms·

What a great historical summary. Compression has moved on now but having grown up marveling at PKZip and maximizing usable space on very early computers, as we

by ctur 3y ago

What a great historical summary. Compression has moved on now but having grown up marveling at PKZip and maximizing usable space on very early computers, as well as compression in modems (v42bis ftw!), this field has always seemed magical. These days it generally is better to prefer Zstandard to zlib/gzip for many reasons. And if you need seekable format, consider squashfs as a reasonable choice. These stand on the shoulders of the giants of zlib and zip but do indeed stand much higher in the modern world.

lxgr 3y ago
> These days it generally is better to prefer Zstandard to zlib/gzip for many reasons. I'd agree for new applications, but just like MP3, .gz files (and by extension .tar.gz/.tgz) and zlib streams will probably be around for a long time for compatibility reasons.
pvorb 3y ago
I think zlib/gzip still has its place these days. It's still a decent choice for most use cases. If you don't know what usage patterns your program will see, zlib still might be a good choice. Plus, it's supported virtually everywhere, which makes it interesting for long-term storage. Often, using one of the modern alternatives is not worth the hassle.
michaelrpeskin 3y ago
I had forgotten about modem compression. Back in the BBS days when you had to upload files to get new files, you usually had a ratio (20 bytes download for every byte you uploaded). I would always use the PKZIP no compression option for the archive to upload because Z-Modem would take care of compression over the wire. So I didn't burn my daily time limit by uploading a large file and I got more credit for my download ratios. I was a silly kid.
- stavros 3y ago
  That sounds like it can be fooled by making a zip bomb that will compress down to a few KB (by the modem), but will be many MB uncompressed. Sounds great for your ratio, and will upload in a few seconds.
- EvanAnderson 3y ago
  That's really clever and likely would have gone unnoticed by a lot of sysops!
  - cpeterso 3y ago
    Another download ratio trick was to use a file transfer client like Leech Modem, an XMODEM-compatible client that would, after downloading the final data block, tell the server the file transfer failed so it wouldn’t count against your download limit. https://en.m.wikipedia.org/wiki/LeechModem https://en.m.wikipedia.org/wiki/LeechModem
    - michaelrpeskin 3y ago
      That's awesome! I totally would have used that as a young punk if I knew about it.

by ctur 3y ago

lxgr 3y ago

> These days it generally is better to prefer Zstandard to zlib/gzip for many reasons. I'd agree for new applications, but just like MP3, .gz files (and by extension .tar.gz/.tgz) and zlib streams will probably be around for a long time for compatibility reasons.

pvorb 3y ago

I think zlib/gzip still has its place these days. It's still a decent choice for most use cases. If you don't know what usage patterns your program will see, zlib still might be a good choice. Plus, it's supported virtually everywhere, which makes it interesting for long-term storage. Often, using one of the modern alternatives is not worth the hassle.

michaelrpeskin 3y ago

I had forgotten about modem compression. Back in the BBS days when you had to upload files to get new files, you usually had a ratio (20 bytes download for every byte you uploaded). I would always use the PKZIP no compression option for the archive to upload because Z-Modem would take care of compression over the wire. So I didn't burn my daily time limit by uploading a large file and I got more credit for my download ratios. I was a silly kid.

stavros 3y ago
That sounds like it can be fooled by making a zip bomb that will compress down to a few KB (by the modem), but will be many MB uncompressed. Sounds great for your ratio, and will upload in a few seconds.
EvanAnderson 3y ago
That's really clever and likely would have gone unnoticed by a lot of sysops!
- cpeterso 3y ago
  Another download ratio trick was to use a file transfer client like Leech Modem, an XMODEM-compatible client that would, after downloading the final data block, tell the server the file transfer failed so it wouldn’t count against your download limit. https://en.m.wikipedia.org/wiki/LeechModem https://en.m.wikipedia.org/wiki/LeechModem
  - michaelrpeskin 3y ago
    That's awesome! I totally would have used that as a young punk if I knew about it.