13 ms·
The Rise and Rise of JSON (2017)
- akubera 6y agoI still think it's a shame that Amazon's Ion format never got widespread adoption. https://amzn.github.io/ion-docs/ https://amzn.github.io/ion-docs/ Superset of JSON with many extra features that people in the comments here desire from their data-language, such as support for comments, timestamps and s-expressions.
- jsnell 6y ago(2017) Previous discussion: https://news.ycombinator.com/item?id=17832936 https://news.ycombinator.com/item?id=17832936
- Zamicol 6y agoJSON is one of the best things to ever happen to software development.
- liberal_098 6y ago> JSON is one of the best things to ever happen to software development Indeed, especially when compared to XML
- abraxas 6y agoIt really isn't. I'm almost tempted to say it's the opposite though that would be overstating the case. It's a format that still bears excessive decoration (what's the purpose of quotes around field names? what are all those commas for?) yet it's limited in the types of data structures that it's able to express (natively). I'm not particularly fond of Clojure specifically but a format like EDN would have been superior in just about every way.
- hackcasual 6y agoI'd hardly call a few bytes per key and field excessive. Especially compared to something like XML. The datastructure complexity being limited is also a pretty significant key to its success. More complex datatypes means greater chances for JSON handling libraries to lack compatibility. The only substantial shortcomings of JSON I see are shortcomings associated with any textual serialization format. Optimizing for human readability in a use case that's 99.99% of the time not read by a human.
- cbsmith 6y agoJSON looks good when compared to XML. That's literally how low you have to go. "The only substantial shortcomings of JSON I see are shortcomings associated with any textual serialization format. Optimizing for human readability in a use case that's not 99.99% of the time not read by a human." There are a few other shortcomings that are reasonably substantial/significant, but yeah, that's the gist of the problem.
- msla 6y ago> JSON looks good when compared to XML. JSON looks good compared to what we'd be using instead of JSON, which is nothing so nice and structured as XML. The competition to JSON is something infinitely more ad-hoc, probably without a distinct parser, such that a generic library to generate or consume it is impossible, and getting usable error messages is equally impossible. > "The only substantial shortcomings of JSON I see are shortcomings associated with any textual serialization format. Optimizing for human readability in a use case that's not 99.99% of the time not read by a human." I agree with this and disagree at the same time: Optimizing for human readability means optimizing for the weird case, the 0.01% (but it seems to be more often than that) of the time you need to go beyond the tools you have to fix something. Saying that's rare is true but inapt: Seatbelts are only used in rare cases, too.
- nitrogen 6y agoIf we didn't have JSON we'd have settled on something like MsgPack.
- cbsmith 6y agoYour 0.01% case is perhaps true with a binary format, but by using JSON, that 0.01% case just became a whole lot larger. ;-) It's like, "hey, we came up with a simple way to do it, but to make it easier to deal with this little edge condition that creates complexity, let's significantly up the complexity and number of edge conditions so they're endemic to the space, and then we're all good go". The irony is, I invariably end up needing to use a computer to help me read JSON anyway.
- kanobo 6y agoQuotes allows for any characters and limited types is probably one of the main reasons it's widely used and implemented. There's no doubt vastly superior formats tailored to specific languages and purposes, but I think it's hard to argue that JSON was not a huge net positive to the software industry as a whole.
- baddox 6y agoListing some specific perceived flaws with JSON isn’t counter to the claim that it’s one of the greatest things to happen in software development.
- fomine3 6y agoLack of integer type is really suck but that's inherited failure from JavaScript.
- redleggedfrog 6y agoIt'd be nice to know what the graph would like like now, three plus years later. I imagine it kept along the same lines. JSON is everywhere. I suspect that a lot of the popularity is piggy-backing on the popularity of JavaScript. But what the heck is .CSV doing on an incline?!
- crooked-v 6y agoCSV generation and parsing is inconsistent based on tooling (see inevitable space/comma/quote problems, for example), while JSON has a strict enough spec that you can unambiguously use it in the same way for CSV-like data as long as nobody needs to literally edit it directly in Excel.
- ciupicri 6y agoCSV has a specification too - RFC 4180, "Common Format and MIME Type for Comma-Separated Values (CSV) Files" [1] [1]: https://tools.ietf.org/html/rfc4180#section-2 https://tools.ietf.org/html/rfc4180#section-2
- goatinaboat 6y agoCSV is so unnecessary - ASCII has built in delimiters for fields outside of the printable characters.
- teddyh 6y agoCSV is for human-readable data. If you don’t need to be human-readable, you might as well go with ASN.1 or something.
- seangrogg 6y agoWait, seriously? Interesting. Can you link me to something that demonstrates that was intended rather than a quirk of the simple format? I always thought it was chosen since it was a seemingly sensible delimiter at first glance and then you only realize it's less useful well after you see others using your spreadsheet app for storing complex strings.
- kanobo 6y agoGreat article. It's strange for me to be old enough to have experienced JSON's entire history as a website maker - I didn't learn anything new in the article I didn't know before and never gave it much thought - but to read it all compiled in one place makes me want to appreciate that we are experiencing tomorrow's history right now.
- amelius 6y agoMy only problem with json files is that you can't add comments to them in a persistent way (especially if changes in the data are written by a program). Unless of course you'd put the comments in the data itself, but that's kind of ugly.
- k__ 6y agoAlso: streaming
- amelius 6y agoCan't you implement streaming by just putting everything in a giant JSON list? Or do you want to send different parts of e.g. a map in separate streams?
- k__ 6y agoHow do you parse a JSON list before you get to the end?
- speedgoose 6y agoYou can use a JSON parser for streams. Instead of returning a full document once everything is parsed, it can process elements one by one.
- Jtsummers 6y agoPresumably similar to how you would stream XML. I guess my question would be, isn't this already possible? You just need a reader that knows that the JSON source is a stream and blocks reads until data is available. Then use a concurrency mechanism to coordinate between the stream reader and the consumer of the stream data to keep things running smoothly (or if you don't mind blocking just use the blocking reader on the same thread/process as the consumer).
- k__ 6y agoHow does this work? If I got multiple items, they're in an array. If such a reader blocks reads to an element until it's finished, I've won nothing if I have to wait until that array finished.
- teddyh 6y ago> no new version of the JSON specification is ever expected to be written. T̵h̵e̵ ̵l̵a̵t̵e̵s̵t̵ ̵v̵e̵r̵s̵i̵o̵n̵ ̵o̵f̵ ̵J̵S̵O̵N̵ ̵w̵a̵s̵ ̵a̵p̵p̵a̵r̵e̵n̵t̵l̵y̵ ̵r̵e̵l̵e̵a̵s̵e̵d̵ ̵l̵e̵s̵s̵ ̵t̵h̵a̵n̵ ̵a̵ ̵y̵e̵a̵r̵ ̵a̵g̵o̵.̵ [EDIT: I was mistaken: the latest standard was RFC 8259. However, this was still published on 2017-12-13, after the article was written.] There have been three different RFCs alone, all defining JSON. (And let’s not even get into the thing called JSON5.)
- seagreen 6y agoWhich specification got an update last year? Google's not turning anything up.
- kozak 6y agoI've tried to find out what was the change less than a year ago, and I couldn't find this information. So, what was the change?
- teddyh 6y agoI seem to have misread my reference: the change a year ago was to Javascript, to make it conform to JSON, not the other way around, which was how I originally interpreted it. The latest version of JSON itself seems instead to be the latest RFC from 2017-12-13. Note: this was still after the article was written.
- throw0101a 6y agoThere were six changes outlined in RFC 8259, three of which were moving from ECMA-262 to make ECMA-404 a normative reference, one was to explicitly state UTF-8 should be used in a particular section, and one was 'to increase the precision of the description of the security risk that follows from using the ECMAScript "eval()" function': * https://tools.ietf.org/html/rfc8259#appendix-A https://tools.ietf.org/html/rfc8259#appendix-A Not very Earth shattering revisions IMHO.
- k__ 6y agoI remember 2010, when I build my first API. People laughed at me for using JSON. I had to add XML and CSV, because "nobody would integrate with a JSON API"
- chc 6y agoI think whatever company you were working at was a few years behind the curve, because JSON was already pretty much the standard for SPAs by 2010. For example, that's the year AngularJS came out, with JSON/JSONP as the only built-in serialization format for communicating with backend APIs.
- k__ 6y agoPretty much, yes. In 2011 we moved from an PHP app to an SPA where JSON came in handy.
- falcolas 6y agoJSON won, IMO, because it's human readable and writable. It came at a time when its main competitor in this space - XML - had gotten too complex. However, JSON is hitting the same limitations and problems that XML faced, and is following in their shoes (namespaces, schemas, x/jpath, implementation drift across libraries, etc).
- krapp 6y agoJSON won because the web and javascript, which supports JSON, won. It would have won regardless of how ugly or awkward it was for that reason alone.
- jjoonathan 6y agoYeah, but they were reinvented in JSON, with the key difference that you didn't have to pay for them if you didn't need them. JSON unbundled XML.
- Spearchucker 6y agoThe same was true for XML.
- jjoonathan 6y agoIt really wasn't. Implicit remote resource dependencies, validation, namespaces, and parser-settings dependence of all of these were huge footguns. You could turn these features off if you controlled the format and parser, but you might not, and in any case you would have to understand the features to ensure you wouldn't trip over them, which meant you couldn't ignore them at zero cost. They were syntactically unbundled but not conceptually unbundled. XML had zero sympathy for the casual user and the results were 100% predictable, earned, and deserved.
- enricozb 6y agoI wish JSON had 1) trailing commas 2) comments (I'm not sure if this is a good idea or not but every once in a while I impulsively write comments)
- kanobo 6y agoIt's funny I have the opposite feeling. In my case I write a formatter/validator in each project for my json that fixes issues like trailing commas and is also where I annotate the data structure, but I can imagine for those who write JSON by hand those two limitations would be a headache.
- tgb 6y agoI just wish it supported all floats (nans and infinities too).
- abraxas 6y ago3) set types, 4) optional schemas 5) multiline strings 6) consistent support for numeric types 7) streaming data The litany is not short and I'm sure I've overlooked a couple.
- selectnull 6y ago8) date types
- speedgoose 6y ago
- raverbashing 6y agoAmen XML is a mess and a chore to work with (at least in any language that isn't Java I guess, but even then). Yes, it has some rough edges. Yes it could be better. But overall it's good. Not too complicated and not too hard. Works fine for most stuff.
- enriquto 6y agoIn 10 years we will see JSON with the same mix of mockery and regret as we see XML today.
- scrollaway 6y agoI remember reading that opinion 10 years ago.
- seagreen 6y agoJust curious-- have you actually read the specifications for both? The difference in length and complexity is like this comment vs. War and Peace.
- atarian 6y agoIf another standard were to dethrone JSON, my guess would be YAML.
- timw4mail 6y agoConsidering that the YAML spec rivals XML in complexity, I doubt it. Between that complexity, incompatible parsers, intentional loose types...what do you really gain?
- jniedrauer 6y agoYAML will never be a sane format for machines to talk to each other. There are just too many ways to interpret it. It's not even really a good human-readable format. Maybe something like TOML? It's very human-readable and simple. But it's not a very good serialization language.
- wvenable 6y agoI remember using XML-RPC back in the day to communicate between a desktop client and a web service. It was fantastic. And then designer went off with a committee somewhere and produced SOAP. The regret with XML is that it continued to evolve into a monstrosity. Thankfully JSON came along and got back to the simplicity of early XML and XML-RPC and stayed there.
- zeveb 6y agoIt's a real shame that both XML and JSON eclipsed the real data format for the ages: S-expressions. I hope it's only temporary, though: there really is no good reason the prefer JSON other than everyone else using it, and if one gets into a novel enough domain then there isn't an everyone else doing anything to worry about yet. S-expressions have the advantage of not containing the redundant object type (there's no need for an object, map or dictionary when one has alists or plists), and the even greater advantage of elegantly representing code. Also they are easier to write parsers for.
- slim 6y agoreal s-expressions end with /s
- teddyh 6y agoThere was this: http://people.csail.mit.edu/rivest/Sexp.txt http://people.csail.mit.edu/rivest/Sexp.txt but it was never even accepted as an RFC, it seems.
- blackrock 6y agoVery interesting. Did anyone else look at this, and think it was super complicated, at first glance? You’d have to be an expert in the data exchange format itself. I can see why Lisp style syntax is bad for human comprehension. It’s because of the lack of an explicit delimiter, like a comma. I don’t think human eyes are very good at using a single space character, as a delimiter. And with this format, you’d have to agree upon the data exchange structure. There would have to be a master key somewhere, maybe as the header. Albeit, this format is far more efficient than XML. But, XML probably won out because it was explicit, and allowed multiple levels of nesting, but at the expense of overly wordy delimiter tags. And JSON made it simpler, by forcing it to be a lightweight key-value pair. It’s too bad something like this didn’t take root and become more popular. This might be a very useful advanced data exchange format for some scenarios. Although the headaches and problems associated with trying to understand, and work with it, might far outweigh its efficiency benefits.
- blackrock 6y ago
- olejorgenb 6y agoA annoying problem with json is the lack of full floating point number support. Transferring NaN, +Inf/-Inf is a pain.
- hajile 6y agoCrockford believes the One True Number should be decimal instead of floating point. He also believes that comments will be abused and turned into ad-hoc parser directives. So everyone gets to suffer for his beliefs.
- velox_io 6y agoI'm quite a big fan of json.org it outlines the JSON specification in a handful of diagrams and a small amount of text, it's so simple and elegant.
- reggieband 6y agoI remember working with SOAP and WSDL. One advantage I kind of miss from that era was the static generators that would create your service classes and the strongly typed DTOs based on the WSDL. I understand some effort has been made to replicate this (e.g. json-schema) but it isn't nearly as widely used as WSDL seemed to be. To be honest, I only kind of miss it since XML was such a major pain to deal with (any one ever had to deal with XPath or XSLT? What a nightmare those were). I recently used GraphQL on a project and it had some nice advantages. I love the idea of protocol buffers but have never had the chance to use them in anger. But if I'm honest, the boring option is JSON and it is what I would use for just about any API I had to expose nowadays.
- 66fm472tjy7 6y agoOne reason in favor of using JSON over XML for web services I don't see mentioned often is that many XML schema based deserializers will by default fail if they get an unexpected element whereas JSON deserializers will ignore it, making it easier to remain backwards-compatible. Example: If you have the following defined in your .xsd <complexType name="MyMethodResponse"> <sequence> <element name="A" type="string" /> <element name="B" type="string" /> </sequence> </complexType> and you later add a "C", existing clients will fail, so you will have to create a new method along side the old one to remain compatible.
- willcipriano 6y agoI've always wondered why the people who designed these things thought that I wanted a exception to be thrown in that case. It seems to be a "enterprise" feature, Java SQL libraries often blow up if they find a column that isn't predefined.
- rahilb 6y agoJSON was clearly built with XGH axioms in mind: ‘errors only appear when you notice them’.
- elcritch 6y agoFollowing proper XML spec always led to more problems than what they were meant to solve. Sometimes I use XML, but just the syntax with a more html5 style to it. Similar to defaulting to error-ing on extra tags, I never got the point of making everything an element and adding extra redundant attributes. That's really just the result of XML that's automatically generated. Like the `<element... type="string">` in your example of classic XML, why not just use `<a>3</a>` or `<a int="3"/>`. XML proper styles still suggest the classic long/autogenerated form. Writing XML in html5 style like `<my-a data-value="3" />` makes it much friendlier. Or as I tend to use it for various internal protocols to wrap multiple CSV data in a file: <some-data type"csv" columns="A, B, C"> 1.3, 3.4, 5.6 1.3, 3.4, 5.6 </some-data> Technically That's really just html5 in a file I guess. :-) But html parsers also don't tend to complain about extra tags too.
- thelazydogsback 6y agoFor small, heterogenous structs like messages and config it makes sense -- for repeated/homogenous collections (array of similar struct - a common case) both JSON and XML are wasteful -- a reflective/inline schema combined with a table would be more readable and space efficient. Also, meta-data needs to be either embedded or provided by special keys, so general meta-data tagging support would be great, where a non-meta-data-aware reader would only get the target data, but any JSON datum could be tagged with another JSON datum as meta-data. (The meta-data tags could be implement the inline-schema for the table as well.)
- The_rationalist 6y agoThis is such a painful contingent historical accident that we have json as the standard instead of hson
- doctoboggan 6y agoThis is a great bit of internet history, if for nothing else than this Douglas Crockford quote in response to a flamebait-y argument about XML being superior to JSON: “The good thing about reinventing the wheel is that you can get a round one.”
- atticusCr 6y agoAlthough it is easier to see the benefits of using JSON as an interchange format because it is lightweight, I still believe that XML is more elegant and verbose than JSON. One of the complains for JSON has been the lack of schemas, although there are some ways around that in projects such as Apache Avro https://avro.apache.org/ https://avro.apache.org/, registry schemas and all that Jazz