5 ms·
Ironically, not long ago there was an article about how hard string handling is in C, so programs tend to be written to avoid manipulating strings as much as po
by userbinator 4y ago
Ironically, not long ago there was an article about how hard string handling is in C, so programs tend to be written to avoid manipulating strings as much as possible. In other languages, where strings are easy to use, it encourages inefficiency like this. The term "stringly typed" also comes to mind.
JSON truly is the new XML, both in terms of advantages and disadvantages. I wish people would stop using it for everything and realise that using a "human-readable" format for data that is 99.999999% not going to be seen by a human is absolute insanity in terms of inefficiency.
- Culonavirus 4y ago> using a "human-readable" format for data that is 99.999999% not going to be seen by a human is absolute insanity in terms of inefficiency. Database dumps say hello. :)
- alex_sf 4y agoIf you want human-readable dumps, there's no reason you can't convert the more-efficient format into something human-readable.
- smittywerben 4y agoI just drag my Windows XML script driver into SYSTEM32 folder (for my HP indigo 7000 series ink press) then I press the rest.
- flandish 4y agoIf you want human readable db output, I might introduce you to what I’ve spent most of my career doing: etl. :)
- afloyd 4y agoHuman readable formats should exist for more or less one purpose, interfacing with a human, eg configuration.
- timcobb 4y agoDeveloping and debugging systems with JSON (vs, say, protocol buffers) is much easier… configuration is rarely interfaced with by humans vs. debug logs or dumps etc, which are always being looked at.
- userbinator 4y agoIt's not hard to read what isn't "human readable" anyway; it just takes a little bit of practice and learning. For example, I can "sight-read" x86 and Z80 Asm from a hexdump, and bits of ASN.1 and a few other proprietary protocols too.
- userbinator 4y agoDid I strike a nerve? It's unfortunate that so many seem to be under the impression that "human readable" is really an euphemism for "like English" and are narrow/closed-minded enough to not consider the possibility that people can learn just about any language with enough exposure and a bit of effort. 4f 6e 63 65 20 79 6f 75 20 73 74 6f 70 20 6c 65 61 72 6e 69 6e 67 2c 20 79 6f 75 20 73 74 61 72 74 20 64 79 69 6e 67 2e
- hyperpape 4y agoYou didn’t strike a nerve, but your comment is misguided. Of course people can read binary formats. It is just that they read them more slowly, and it requires quite a bit of exposure/practice, so that people usually invest in tooling to simplify the process. I suggest that when you see a comment like this, you ask yourself “if I change ‘human readable’ to ‘easily human readable’ would I need to offer my correction?”
- code_biologist 4y agoYour comment comes across the same way as that long ago "Show HN" where somebody said that Dropbox is pretty much just rsync.
- catiopatio 4y ago
- timcobb 4y agoBut in systems where JSON is used inefficiently, it’s rarely (with some notable exceptions [0]) the a performance limiting factor. [0] https://news.ycombinator.com/item?id=26296339 https://news.ycombinator.com/item?id=26296339
- pohuing 4y agoThat one wasn't even down to json, but rather some c stdlib functions being in linear time because c strings(which then compounded into quadratic) right?
- ilyt 4y agoThat's a massive exaggeration. XML is both much more complex (and bug-prone) to parse, and wastes more space. And the problem here wasn't JSON in the first place anyway, it was > The “back” stack is unlimited, meaning we’re saving more and more state until we explode And storing previous state in serialized string in current state, instead of just having it stored natively. You'd have same problem if you tried to serialize <format> into a field in <format>
- userbinator 4y agoYou'd have same problem if you tried to serialize <format> into a field in <format> Length-delimited formats don't need (exponentially increasing) escaping.
- deepsun 4y agoBut why do you bring XML? The parent is against human-readable formats, and XML is one of them.
- simplotek 4y ago> And the problem here wasn't JSON in the first place anyway Yes it was. They stored state in a huge JSON dump which ended up working like a zip bomb by the way they had to escape a string. This alone resulted in an exponential increase in memory that quickly became a problem. Think about it for a second: would they experience this issue if they stored an in-memory stack per user? They wouldn't. Heck, it seems that they could live with their JSON choice if they replaced '/' with any other inescapable character and converted it back to '/' when reading.
- xigoi 4y agoXML would also have a problem with this: you'd replace < with <, then with &lt;, then with &amp;lt;, … It's not exponential, but still inefficient.
- simplotek 4y agoInefficient is not the same as problematic. They turned what otherwise would be a linear increase in memory usage into an exponential increase, which expectedly resulted in memory exhaustion problems. There are plenty of problems to go around this approach and overall system design, but the weakest link is undoubtedly how JSON was used.
- berkes 4y agoI'd argue that JSON is one of the least Human Readable serialization formats. It lacks comments. Has no way to offer context. Cannot impose limitations, and normalizing needs additional, often poorly supported RFCs (JSON-LD or pointers). It also is a poor non-human readable format, as it lacks crucial types (datetime, decimal) is rather verbose (just imagine the terabytes bandwidth wasted, globally on all these commas and curly braces) and cannot be parsed streaming.
- codeflo 4y ago1. I 100% agree about comments. 2. I’m still not convinced it’s the file format’s job to encode every possible data type. We used to have deserialization layers for that, but for some reason, we don’t do that anymore. Nowadays we expect every file format to be self-describing, which I think is wasteful and limiting. But that’s a matter of opinion, obviously. 3. What do you mean by “cannot be parsed streaming”? Or to be perhaps more precise, what’s missing in existing streaming JSON parsers? I haven’t used any of them yet.
- mxmlnkn 4y agoMany JSON parsers require the whole JSON file to be in memory. Parsers that allow parsing in streaming manner are the exception. I only found jsoncons to work reliable but at the cost of speed compared to something fast like rapidjson. And even jsoncons might have the problem that large strings have to be fully in memory, i.e., if someone has the fun idea to store a 1 GB string inside a JSON, then you still have a potential out-of-memory event at your hand. There are other attempts to fix this problem with the jsonl format but that format feels so ad-hoc that I'm not convinced yet. It stores one JSON object per line. Of course, this comes with its own problems such as each JSON object potentially still being unbounded and, now that lines are used to separate JSON objects, you cannot format those JSON objects in a human-readable way anymore. Also, the first time I encountered a file like this, it simply had a .json extension and I was wondering why every JSON parser failed to parse that file. Because it isn't valid JSON and should have a separate extension.
- 4y ago
- jffhn 4y ago>using a "human-readable" format for data that is 99.999999% not going to be seen by a human is absolute insanity in terms of inefficiency. In the encode/decode library we use, the generated toString were accidentally so close to json, that we made them exactly so (mostly had to add quotes). Now we can use json tooling on top of efficient messaging.