3 ms·
Besides ZIP metadata, there may be flushes in specific points in the deflate stream (which applies to .gz files as well). These flushes reset the compression di
by mkup 4y ago
Besides ZIP metadata, there may be flushes in specific points in the deflate stream (which applies to .gz files as well). These flushes reset the compression dictionary and make further compressed data independent from previous data (at the expense of losing some compression efficiency). So: AWS S3 customer may have injected these flushes to their .gz files (for whatever reason, e.g. steganography), and after gzip-to-zstd-to-gzip transcompression this steganographic data will be be lost (and of course sha256 and other similar hashes will be different, as you already said).
- sgtnoodle 4y agoExactly. I've built several logging systems over the years that intentionally flush compression streams (or concatenate gz streams) for robustness reasons.