3 ms·

I get the same ordering with gzip, but it seems sensitive to compression algorithm. With bzip2 (default settings, v1.0.6), homogenius seems to actually worsen t

I get the same ordering with gzip, but it seems sensitive to compression algorithm. With bzip2 (default settings, v1.0.6), homogenius seems to actually worsen the compression ratio, at least on this one example: 52093 verbose.json.bz2 56298 packed-homogenius.json.bz2 68521 packed-homogenius.json.gz 79222 verbose.json.gz 213742 packed-homogenius.json 265177 verbose.json

Scaevolus 12y ago
BWT-based compressors like bzip2 do best when their inputs have highly repetitive structure. In a JSON file with many repeated keys, the information that `"k` is usually followed by `ey_1": "` is compressed very effectively. For similar reasons, bzip2 tends to outperform gzip on executables-- it can more effectively model the conditional probability of opcode sequences. See http://mattmahoney.net/dc/dce.html#Section_55 http://mattmahoney.net/dc/dce.html#Section_55 for more discussion