5 ms·
Base64 has been around forever, and hex has significantly lower information density than it - consistently more than 30% smaller. As for doing something horrib
by _wmd 9y ago
Base64 has been around forever, and hex has significantly lower information density than it - consistently more than 30% smaller.
As for doing something horribly wrong, I dunno, ensuring literally billions of moms and pops can exchange baby pictures by e-mail, for the first time in human history, and doing so since the time when we were using 33.6kbit modems, that seems like a resounding success.†
† Yes, if e-mail had not paid such close attention to compatibility, and just ripped up every old standard on every chance it got (i.e. pretty much like any technology invented since the commoditization of the web), interoperability would never have happened. Legacy crap is fundamentally a part of that success, and Base64 is fundamentally a part of making those hacks practical for the technology we had at the time††
†† To head off an obvious counterargument: yes, there are contemporary examples of why this strategy is still important (it's, so far, the only one that has ever worked in the decentralized environment of the Internet), and is very likely to come in useful in future, time and time again
- tzahola 9y agoMost of the modern Web is unusable without a highly optimized JIT-compiling JS interpreter. Are you sure about the importance of interoperability & compatibility?
- _wmd 9y agoI'm not sure why you believe that is a counterexample - it is anything but. That interpreter can still run code written against the original DOM APIs in the original syntax, as defined 22 years ago - around time the <noscript> tag appeared in HTML! Of course if mom and pop were using UUCP and an ADM-3A to fetch their mail, they couldn't view the images -- but all the infrastructure that allowed them to read mail at all could be repurposed without change, possibly including their modem (thanks, RS-232, introduced 1960), if they bought a 80386 desktop with a VGA card These are all interoperability success stories! Meanwhile, I need 19 different IM apps on my phone to talk to 19 different people. That's what happens when you ignore interoperability
- mschuster91 9y ago> Meanwhile, I need 19 different IM apps on my phone to talk to 19 different people. That's what happens when you ignore interoperability IO was not ignored, it was willfully cast aside. That's the world of walled gardens we live in today... federation makes it possible for users to switch outside your walled garden.
- ycombobreaker 9y agoYes, HTTP and all of the layers below are quite fundamental to the success of the WWW. Interoperability breeds competition and real innovation because nobody can lean on a network effect the way modern walled gardens do.
- hackcasual 9y agoHex has lower information density, however it is more compressible, particularly if your data is byte aligned. For example the string "aaaaaa" is b64 encoded as "YWFhYWFh" but in hex is "616161616161". On a project I'm on, we switched from base64 to hex for some binary data embedded in JSON, and saw ~20% size reduction, since its always compressed by the web server.
- _wmd 9y agoToday I learned! >>> import random,zlib,codecs >>> sample = random.getrandbits(1024).to_bytes(1024, 'big') * 1024 >>> len(zlib.compress(codecs.encode(sample, 'base64'))) 22083 >>> len(zlib.compress(codecs.encode(sample, 'hex'))) 4326 >>> edit: (25 minutes of digging around the Internet, and I find this): >>> sample = os.urandom(1024) * 1024 >>> len(sample.encode('base64')) 1416501 >>> len(sample.encode('hex')) 2097152 >>> len(sample.encode('ascii85')) 1310720 >>> len(zlib.compress(sample.encode('base64'))) 82169 >>> len(zlib.compress(sample.encode('hex'))) 12537 >>> len(zlib.compress(sample.encode('ascii85'))) 8212 Had to use Python 2.x to make use of the 'hackercodecs' package that implements ascii85, but looks like it's the best of both worlds, assuming its character set suits whatever medium you are transferring over, and decoding it on the other end doesn't require some slow code. final edit: I'm guessing it was an accidental trick of the repetitive input data lining up well. On real data hex still wins out (which should have been obvious in hindsight): >>> sample = ''.join(sorted(open('/usr/share/dict/words').readlines())) >>> len(zlib.compress(sample.encode('ascii85'))) 1320809 >>> len(zlib.compress(sample.encode('base64'))) 1116678 >>> len(zlib.compress(sample.encode('hex'))) 880651
- andreareina 9y agoHuh, my results are drastically different: $ dd if=/dev/urandom bs=512 count=2048 | base64 | gzip | wc ... 4088 31928 1059085 $ dd if=/dev/urandom bs=512 count=2048 | xxd -p | gzip | wc ... 5019 33798 1231268 1 megabyte of random data consistently results in ~1 megabyte of compressed base64 text, ~1.2 megabytes of compressed hex.