6 ms·

2006, well into the era of XML being the trendy fad that every piece of Serious Business software was supposed to use. Now 18 years later, JSON seems to have d

2006, well into the era of XML being the trendy fad that every piece of Serious Business software was supposed to use. Now 18 years later, JSON seems to have displaced it. Personally, I've never found text-based formats to be a good choice for data that humans will rarely need to read or write; I much prefer simple and efficient binary formats, which can be just as extensible without the additional inefficiency and needlessly-introduced-failures of string handling.

joshlk 3y ago
I’m sure it’s down to use case and circumstances but I frequently find it useful to be able to read and edit text based data formats (JSON/YAML/TOML). If you are concerned about data size you can easily use on-the-fly compression at little cost nowadays.
devjab 3y ago
In my experience people read the JSON output of things more often than anything else in both development and in maintenance. You can certainly make a point about that being stupid, and it certainly also is stupid. It just seems to be what happens when you have a gazillion APIs and even more business processes and a lot of people who should know what they are doing, who don’t. I recently had the joy of reverse engineering a terrible piece of JavaScript which created a vast and ridiculously complicated JSON delivery for one of our most important frontends. It was so full of any assumptions that the original author had several functions which were still doing various things with “unknown” because the data which had once been there wasn’t. Anyway, since I had to reverse engineer it from terrible code and no documentation I figured I’d just talk with the people who worked the data inputs and outputs and maybe learn something about the process from their business processes. Turns out they couldn’t help, what was even worse was how the frontend partner we were delivering the JSON to couldn’t even explain why they needed it the way it arrived. Those were some “fun” sessions trying to Sherlock Holmes out of the mess. This was a bad story, but it’s just one out of a thousand such stories in enterprise and non-tech IT. I think the reason JSON “won” over XML was the ”same” human reason. I’ve seen a lot of XML, but I don’t think I’ve ever seen some that didn’t break its own standards. Like you’d get all XML with data between ><‘s and then suddenly a single field would have the data inside the <… value=“”>… because duck you, or whatever else. Often they wouldn’t bother supplying you with schemas, or if they did, their auto-generated schemas wouldn’t have been updated in five years because apparently someone forgot that. Then once you “progress” beyond the technical funzies you’ll still have to deal with using the XML to find out what the hell is going on, and since it’s much harder to read (not entirely sure why) you’ll have to first convert it into JSON anyway, before whatever business process person has to read it.
- Pamar 3y ago
  Sad applause...
- dfgdfg34545456 3y ago
  The decision to lock down JSON without comments was a good one, stopped all these proxy languages developing in it, like what happened with XML.
- dfgdfg34545456 3y ago
  With you, but just have to add that the decision to return JSON by LSP protocol to code editors is simply ridiculous.
eru 3y ago
> Personally, I've never found text-based formats to be a good choice for data that humans will rarely need to read or write; I much prefer simple and efficient binary formats, which can be just as extensible without the additional inefficiency and needlessly-introduced-failures of string handling. Many modern serialisation / deserialisation libraries (like Rust's serde) let you serialise your data structures to a variety of formats, including both JSON and various binary formats of your choice. But it should also be relatively easy to take any binary format, and define an alternate text based format for it that allows one-to-one conversion in both directions. A bit like assembly vs binary machine language.
mrweasel 3y ago
> I've never found text-based formats to be a good choice for data that humans will rarely need to read or write; A computer may need to read the data millions of times, but humans only need to read it when things goes wrong. If you take the route of using a binary format, then you need to provide a robust set of tools to view, edit, debug that format. It can absolutely work, journald is binary, but has the tooling to help you get the plain text, when you need it. Protobuf could work well, but if you design your own binary data format you have to build some tooling as well. For many (most) it doesn't matter much if you use XML or JSON, the computer will process it fast enough. If the concern is parsing speed, then yes, absolutely binary formats might be the best option. For things like HFT or anything else that processes vast volumes of data, its probably the only choice.
- userbinator 3y ago
  I can read from a hexdump a few binary formats I've worked with over the years. Not really harder than learning any other language, but much more concise than text-based formats.
  - mrweasel 3y ago
    Given that I don't know what you do, that might be a completely reasonable solution, in your case. My background is being woken up at 3AM by alert of call for systems where I have less that 100% familiarity and is asked to troubleshot. Looking at a hexdump for a format I don't know is not all that helpful. Seeming a mangled XML document, or reading the JSON output from an API call can frequently be enough to know if I need to call a developer, or if I can work with the client and fix the data. Binary formats are absolutely a valid solution of a number of problems and if it works in your context there's no reason to change it.
- anonzzzies 3y ago
  For most things, SQLite is a very good format and somehow still underused in favour of custom formats (without or with very basic) tooling.
  - 3y ago
rlv-dan 3y ago
A problem with binary formats is not now, it's tomorrow when tools have changed and documentation is lost (if there ever was any).
eggdaft 3y ago
The Pragmatic Programmer explains why plain text is the way to go. Biggest issue with binary is the difficulty reading / manipulating it long-term.
dfgdfg34545456 3y ago
Configuration files as text are nicer in source control.
tannhaeuser 3y ago
> Now 18 years later, JSON seems to have displaced it. More like, the people who didn't understand XML was designed to simplify HTML's SGML serialization moved elsewhere to misuse JSON (and YAML, and other things), the use cases and limitations of which they also failed to understand. From the XML spec (edited by Tim Bray, the author of the linked blog): > XML is a subset of SGML that is completely described in this document. Its goal is to enable generic SGML to be served, received, and processed on the Web in the way that is now possible with HTML. [1] [1]: https://www.w3.org/TR/1998/REC-xml-19980210.html https://www.w3.org/TR/1998/REC-xml-19980210.html
- fmajid 3y ago
  And yet they didn’t simplify enough, keeping unexplored land mines of no utility whatsoever like XXE that keep yielding CVEs.
DrDroop 3y ago
The funny thing about this discussing is that no one seems to care about the actual problem that is being solved by XML rather then just serialising and deserialising some data used by one piece of software. XML is mainly useful for exchanging data between organisations in a pre agreed format and schema where everyone is doing something else with the information. I'm implementing XBRL right now, and I can't say that I enjoy it, but I also wouldn't know how to do it better except maybe in some superficial way. Sure XML is bloated and maybe something like RDF/TTL would be better because of build in support for links, multiple languages and datatypes but that still leaves the problem of having to deal with whatever is expressed in a taxonomy that can be GB big.
tootie 3y ago
On the flip side, XML is by far the superior choice for documents people will have to read. Paired with XSD it makes it a lot easier to created valid, well-formed documents that conform to expected input. Ideal for anything that isn't required to be transmitted over the wire frequently. Like config files. I feel like we could save the world thousands of hours of head-scratching if things like web server config or deployment descriptors were done in rigorously-defined XML formats instead of made-up formats like Dockerfile.
nocombination 3y ago
Generally there is misunderstanding of these markups. JSON came along because the JavaScript people found it convenient and more network-efficient (the irony being it's not really). But a million years later followed the schema-validation logic to back-fill deficiencies. In my opinion, one should go for XML when writing a portable document format of any variety to allow a vast array of schema validators, linters, and so-forth. This makes writing plugins which target a given schema much more portable and easier to write. It's kind of ridiculous how many times web folks reinvent the wheel. Seriously, get rid of YAML, TOML, JSON. We already had INI, XML/XPATH/XSD. There's nothing wrong with XML, people are just lazy and so is JSON. It's the lazy, sloppy cousin of XML which was a well-thought-out standard to fill a deficiency in HTML for data. Honestly though, we should be using something like this: http://openddl.org/ http://openddl.org/
- messe 3y ago
  > Seriously, get rid of YAML, TOML, JSON. We already had INI, XML/XPATH/XSD I agree with you on YAML/JSON, but isn't TOML more or less INI with more data types and an actual specification?
  - nocombination 3y ago
    Yeah sorry didn't mean to add TOML in there. And to be fair, YAML is somewhat clean for what it is. I shouldn't have included that in my rant either. My main gripe is with the "JSON Everywhere" approach. :) There's no one-size-fits-all solution to anything. If one is using JavaScript—great, use JSON as a way to transmit objects over the wire or unpack an object into a template. Just PLEASE do not use it for a configuration format. Visual Studio Code configuration is a nightmare of ugliness.
- dtech 3y ago
  The fact that XML External Entity Processing is a category of vulnerability indicates how wrong your comment is. There's a lot of things wrong with XML, mostly that it's hugely complicated and a nightmare to write parsers for. JSON won because all of that complexity wasn't in it nor needed.
- 3y ago
_a_a_a_ 3y ago
Easy human readability is a massive win when you have to diagnose problems.

joshlk 3y ago

I’m sure it’s down to use case and circumstances but I frequently find it useful to be able to read and edit text based data formats (JSON/YAML/TOML). If you are concerned about data size you can easily use on-the-fly compression at little cost nowadays.

devjab 3y ago

In my experience people read the JSON output of things more often than anything else in both development and in maintenance. You can certainly make a point about that being stupid, and it certainly also is stupid. It just seems to be what happens when you have a gazillion APIs and even more business processes and a lot of people who should know what they are doing, who don’t. I recently had the joy of reverse engineering a terrible piece of JavaScript which created a vast and ridiculously complicated JSON delivery for one of our most important frontends. It was so full of any assumptions that the original author had several functions which were still doing various things with “unknown” because the data which had once been there wasn’t. Anyway, since I had to reverse engineer it from terrible code and no documentation I figured I’d just talk with the people who worked the data inputs and outputs and maybe learn something about the process from their business processes. Turns out they couldn’t help, what was even worse was how the frontend partner we were delivering the JSON to couldn’t even explain why they needed it the way it arrived. Those were some “fun” sessions trying to Sherlock Holmes out of the mess. This was a bad story, but it’s just one out of a thousand such stories in enterprise and non-tech IT. I think the reason JSON “won” over XML was the ”same” human reason. I’ve seen a lot of XML, but I don’t think I’ve ever seen some that didn’t break its own standards. Like you’d get all XML with data between ><‘s and then suddenly a single field would have the data inside the <… value=“”>… because duck you, or whatever else. Often they wouldn’t bother supplying you with schemas, or if they did, their auto-generated schemas wouldn’t have been updated in five years because apparently someone forgot that. Then once you “progress” beyond the technical funzies you’ll still have to deal with using the XML to find out what the hell is going on, and since it’s much harder to read (not entirely sure why) you’ll have to first convert it into JSON anyway, before whatever business process person has to read it.

Pamar 3y ago
Sad applause...
dfgdfg34545456 3y ago
The decision to lock down JSON without comments was a good one, stopped all these proxy languages developing in it, like what happened with XML.
dfgdfg34545456 3y ago
With you, but just have to add that the decision to return JSON by LSP protocol to code editors is simply ridiculous.

eru 3y ago

> Personally, I've never found text-based formats to be a good choice for data that humans will rarely need to read or write; I much prefer simple and efficient binary formats, which can be just as extensible without the additional inefficiency and needlessly-introduced-failures of string handling. Many modern serialisation / deserialisation libraries (like Rust's serde) let you serialise your data structures to a variety of formats, including both JSON and various binary formats of your choice. But it should also be relatively easy to take any binary format, and define an alternate text based format for it that allows one-to-one conversion in both directions. A bit like assembly vs binary machine language.

mrweasel 3y ago

> I've never found text-based formats to be a good choice for data that humans will rarely need to read or write; A computer may need to read the data millions of times, but humans only need to read it when things goes wrong. If you take the route of using a binary format, then you need to provide a robust set of tools to view, edit, debug that format. It can absolutely work, journald is binary, but has the tooling to help you get the plain text, when you need it. Protobuf could work well, but if you design your own binary data format you have to build some tooling as well. For many (most) it doesn't matter much if you use XML or JSON, the computer will process it fast enough. If the concern is parsing speed, then yes, absolutely binary formats might be the best option. For things like HFT or anything else that processes vast volumes of data, its probably the only choice.

userbinator 3y ago
I can read from a hexdump a few binary formats I've worked with over the years. Not really harder than learning any other language, but much more concise than text-based formats.
- mrweasel 3y ago
  Given that I don't know what you do, that might be a completely reasonable solution, in your case. My background is being woken up at 3AM by alert of call for systems where I have less that 100% familiarity and is asked to troubleshot. Looking at a hexdump for a format I don't know is not all that helpful. Seeming a mangled XML document, or reading the JSON output from an API call can frequently be enough to know if I need to call a developer, or if I can work with the client and fix the data. Binary formats are absolutely a valid solution of a number of problems and if it works in your context there's no reason to change it.
anonzzzies 3y ago
For most things, SQLite is a very good format and somehow still underused in favour of custom formats (without or with very basic) tooling.
- 3y ago

rlv-dan 3y ago

A problem with binary formats is not now, it's tomorrow when tools have changed and documentation is lost (if there ever was any).

eggdaft 3y ago

The Pragmatic Programmer explains why plain text is the way to go. Biggest issue with binary is the difficulty reading / manipulating it long-term.

dfgdfg34545456 3y ago

Configuration files as text are nicer in source control.

tannhaeuser 3y ago

> Now 18 years later, JSON seems to have displaced it. More like, the people who didn't understand XML was designed to simplify HTML's SGML serialization moved elsewhere to misuse JSON (and YAML, and other things), the use cases and limitations of which they also failed to understand. From the XML spec (edited by Tim Bray, the author of the linked blog): > XML is a subset of SGML that is completely described in this document. Its goal is to enable generic SGML to be served, received, and processed on the Web in the way that is now possible with HTML. [1] [1]: https://www.w3.org/TR/1998/REC-xml-19980210.html https://www.w3.org/TR/1998/REC-xml-19980210.html

fmajid 3y ago
And yet they didn’t simplify enough, keeping unexplored land mines of no utility whatsoever like XXE that keep yielding CVEs.

DrDroop 3y ago

The funny thing about this discussing is that no one seems to care about the actual problem that is being solved by XML rather then just serialising and deserialising some data used by one piece of software. XML is mainly useful for exchanging data between organisations in a pre agreed format and schema where everyone is doing something else with the information. I'm implementing XBRL right now, and I can't say that I enjoy it, but I also wouldn't know how to do it better except maybe in some superficial way. Sure XML is bloated and maybe something like RDF/TTL would be better because of build in support for links, multiple languages and datatypes but that still leaves the problem of having to deal with whatever is expressed in a taxonomy that can be GB big.

tootie 3y ago

On the flip side, XML is by far the superior choice for documents people will have to read. Paired with XSD it makes it a lot easier to created valid, well-formed documents that conform to expected input. Ideal for anything that isn't required to be transmitted over the wire frequently. Like config files. I feel like we could save the world thousands of hours of head-scratching if things like web server config or deployment descriptors were done in rigorously-defined XML formats instead of made-up formats like Dockerfile.

nocombination 3y ago

Generally there is misunderstanding of these markups. JSON came along because the JavaScript people found it convenient and more network-efficient (the irony being it's not really). But a million years later followed the schema-validation logic to back-fill deficiencies. In my opinion, one should go for XML when writing a portable document format of any variety to allow a vast array of schema validators, linters, and so-forth. This makes writing plugins which target a given schema much more portable and easier to write. It's kind of ridiculous how many times web folks reinvent the wheel. Seriously, get rid of YAML, TOML, JSON. We already had INI, XML/XPATH/XSD. There's nothing wrong with XML, people are just lazy and so is JSON. It's the lazy, sloppy cousin of XML which was a well-thought-out standard to fill a deficiency in HTML for data. Honestly though, we should be using something like this: http://openddl.org/ http://openddl.org/

messe 3y ago
> Seriously, get rid of YAML, TOML, JSON. We already had INI, XML/XPATH/XSD I agree with you on YAML/JSON, but isn't TOML more or less INI with more data types and an actual specification?
- nocombination 3y ago
  Yeah sorry didn't mean to add TOML in there. And to be fair, YAML is somewhat clean for what it is. I shouldn't have included that in my rant either. My main gripe is with the "JSON Everywhere" approach. :) There's no one-size-fits-all solution to anything. If one is using JavaScript—great, use JSON as a way to transmit objects over the wire or unpack an object into a template. Just PLEASE do not use it for a configuration format. Visual Studio Code configuration is a nightmare of ugliness.
dtech 3y ago
The fact that XML External Entity Processing is a category of vulnerability indicates how wrong your comment is. There's a lot of things wrong with XML, mostly that it's hugely complicated and a nightmare to write parsers for. JSON won because all of that complexity wasn't in it nor needed.
3y ago

_a_a_a_ 3y ago

Easy human readability is a massive win when you have to diagnose problems.