4 ms·
That doesn't work, since there's no way to distinguish a compressed file from an uncompressed file that just happens to have the same contents. The hash doesn't
by svara 5y ago
That doesn't work, since there's no way to distinguish a compressed file from an uncompressed file that just happens to have the same contents. The hash doesn't help with that since that has collisions, it's the same pigeon hole issue.
Most files in the space of all possible files will be essentially random and not compress, and adding a header will make them larger.
- willvarfar 5y agoA compressed file is distinguished by being a valid gzip file with a valid hash. Such a file won’t compress a second time, so the length will remain unchanged. Then, decompressing twice will also generate the same output.
- comex 5y agoThen what happens if the user starts with a file that happens to be a valid gzip file with a valid hash, compresses it twice, and decompresses it twice?
- nuccy 5y agoOk, then what if we save some space for the single bit indicator (compressed/non compressed) by following a simple algorithm: 1. Look for at least two as long as possible identical peaces with arbitrary bit offset from the beginning. 2. Record value and length (common for all peaces) and offsets for each peace. 3. Create and prepend a header with following fields: compressed_flag, piece length, pieces count, piece value, pieces offsets list, content Obviously if the file is very small this approach will not work. But assuming that we know the file size (which is the loophole here), we can pre-calculate which is a threshold file size to consider to be with a header, and what is too short so uncompressed headerless content is being saved there. P.S. To remove the loophole possibility the streams of data should be rather considered instead of files.