3 ms·
I wonder if gzip has improved? 15 years ago when I tested this, it was more efficient to gzip base16 encoded data than base64 encoded data. (At least, the Engli
by NelsonMinar 8y ago
I wonder if gzip has improved? 15 years ago when I tested this, it was more efficient to gzip base16 encoded data than base64 encoded data. (At least, the English dictionary.) I assume that was because the 3:4 encoding broke up patterns in the source text and messed with the compressor.
But I just tested this again and it's not true anymore.
$ wc -c american-english
971578 american-english
$ gzip < american-english | wc -c
259977
$ base64 < american-english | gzip | wc -c
429263
$ hexdump -e '"%x"' < american-english | gzip | wc -c
478411
- kbaker 8y agoActually, it is still true. hexdump has some confusing format options... what you were using was converting it to little-endian first before printing the hex representation, which really messed with gzip. Try this: $ hexdump -e '"%x"' < american-english | gzip | wc -c 463871 $ hexdump -v -e '/1 "%02x"' < american-english | gzip | wc -c 302515 $ base64 < american-english | gzip | wc -c 415737
- eadmund 8y ago> converting it to little-endian first Ack, is there anything little-endian doesn't ruin? /me misses 68000 assembler …
- NelsonMinar 8y agoThanks! That'll teach me to cut and paste from StackExchange. Also why I got different results last time I looked at this; was using a different base 16 encoder.