5 ms·

Why are we passing around binary data re-encoded to only use 6 bits per transmitted byte? In some cases it might be practical to shoehorn binary data where a te

Why are we passing around binary data re-encoded to only use 6 bits per transmitted byte? In some cases it might be practical to shoehorn binary data where a text string has historically gone (e.g., data URLs, emails) but making a new format that can’t handle binary data without costly armoring seems like we’ve basically given up on using the right tool for the job.

AnIdiotOnTheNet 7y ago
> seems like we’ve basically given up on using the right tool for the job That's webtech in a nutshell isn't it?
emj 7y ago
Cognitive load. A readable dataformat is easy to understand, binary is not readable. The thing is standards for formats get broken all the time, and a broken CSV files is usually easier to fix than a non-standard binary format.
- londons_explore 7y ago
  The same argument could have been made when the ppm image format was replaced with binary formats like JPEG. Nobody says "what a shame I can't open that jpeg image in a text editor". Everyone understands that an image is an image, and there are tools for dealing with images, like photoshop. The same could be said for data interchange formats like json.
  - emj 7y ago
    This was specifically about having a fileformat encoding binary data, and in that vein there is a place for PPM, it's probably better for many small bitmaps than PNG. My old PPMs from my email archive look horrible as jpegs https://xkcd.com/1683/ https://xkcd.com/1683/ . JSON vs. Bencode, JSON can get you the data good enough a lot faster, while a bencode parser is easier to write. My feeling is that most formats with schemas usually fail somewhere in the details anyways. Honestly I think I want too much from them; e.g. the YANG Data model language is a kitchen sink but lacks a lot.
  - yellowapple 7y ago
    > Nobody says "what a shame I can't open that jpeg image in a text editor". Nobody except me, of course. I'd be thrilled if I could open up a JPEG in Emacs and e.g. view/edit the EXIF metadata in a text buffer (and/or pop up a graphics buffer for editing the image itself). Similarly, I'd be thrilled if Emacs was able to parse MessagePack data and let me view/edit it in a buffer. Both of these things are theoretically possible, but to my knowledge nobody's actually done them yet. Granted, calling Emacs "just" a text editor is pretty stretchy, but still.
bobbyi_settv 7y ago
> Why are we passing around binary data re-encoded to only use 6 bits per transmitted byte? We're not. We're gzipping the json when we transmit it.

AnIdiotOnTheNet 7y ago

> seems like we’ve basically given up on using the right tool for the job That's webtech in a nutshell isn't it?

emj 7y ago

Cognitive load. A readable dataformat is easy to understand, binary is not readable. The thing is standards for formats get broken all the time, and a broken CSV files is usually easier to fix than a non-standard binary format.

londons_explore 7y ago
The same argument could have been made when the ppm image format was replaced with binary formats like JPEG. Nobody says "what a shame I can't open that jpeg image in a text editor". Everyone understands that an image is an image, and there are tools for dealing with images, like photoshop. The same could be said for data interchange formats like json.
- emj 7y ago
  This was specifically about having a fileformat encoding binary data, and in that vein there is a place for PPM, it's probably better for many small bitmaps than PNG. My old PPMs from my email archive look horrible as jpegs https://xkcd.com/1683/ https://xkcd.com/1683/ . JSON vs. Bencode, JSON can get you the data good enough a lot faster, while a bencode parser is easier to write. My feeling is that most formats with schemas usually fail somewhere in the details anyways. Honestly I think I want too much from them; e.g. the YANG Data model language is a kitchen sink but lacks a lot.
- yellowapple 7y ago
  > Nobody says "what a shame I can't open that jpeg image in a text editor". Nobody except me, of course. I'd be thrilled if I could open up a JPEG in Emacs and e.g. view/edit the EXIF metadata in a text buffer (and/or pop up a graphics buffer for editing the image itself). Similarly, I'd be thrilled if Emacs was able to parse MessagePack data and let me view/edit it in a buffer. Both of these things are theoretically possible, but to my knowledge nobody's actually done them yet. Granted, calling Emacs "just" a text editor is pretty stretchy, but still.

bobbyi_settv 7y ago

> Why are we passing around binary data re-encoded to only use 6 bits per transmitted byte? We're not. We're gzipping the json when we transmit it.