9 ms·
Deprecating XML
- haberman 16y agoXML's bad, JSON is better. But Protocol Buffers are best. Like JSON, the data model for Protocol Buffers maps nicely onto simple data structures (unlike DOM). But with Protocol Buffers you also get a schema for free, a wicked efficient binary format if you want it, default values, etc. And you can use JSON as a text format. In other words, you could take your JSON that you have sitting around, whip up a Protocol Buffers .proto file for it, and get nice generated C++ classes for it with full schema validation. Your JSON file: {field1="foo", field2=5} ...could be accessed from a C++ object as: my_obj.field1(); The only bummer about protocol buffers is that its support for high-level languages (PHP, Perl, Ruby, etc) is not very good. I've been working on a separate implementation of Protocol Buffers in C to address this (by making it easy to write bindings for) but this project has unfortunately been stalled as I've been busy with work and life. :(
- alavrik 16y agoWhat is it the tool that generates C++ codecs for JSON from .proto specifications? (I wrote a generic command-line Protobuf-JSON converter).
- haberman 16y agoHmm, I just looked and apparently this code isn't part of the open-source protobuf release. :( This code exists inside Google, but apparently hasn't been disentangled from the rest of the Google source tree to be released outside.
- prodigal_erik 16y agoThe protobuf encoding carries a ton of redundant crap around (if not as much as text formats). You can do a lot better by paying more attention to the schema and omitting facts that aren't allowed to change, like tags on self-delimiting required fields and arrays (ASN.1 PER does this well). The protobuf tradeoffs only make any sense if you think you can somehow do something useful with messages that are somewhat but not entirely corrupted, because you still have to solve the problem of finding intact field boundaries without being given any HDLC-style framing.
- haberman 16y ago> The protobuf encoding carries a ton of redundant crap around (if not as much as text formats). [citation needed] One of the features of protocol buffers is that they're backwards and forwards compatible. You can add and remove fields, change "required" to "optional" and back again, and still make sense of what comes to you on the wire. I don't think there's much of anything that can be eliminated from the protobuf binary format. Eliminating tags for a field (even if you consider it required) wouldn't be backward compatible with a previous version of the protocol that considered it optional. A message from a previous version of the schema is not "corrupted." Being able to make wire-compatible changes to the protocol is an extremely important feature.
- prodigal_erik 16y agoYou can't add or remove required fields, because outdated recipients will be dangerously wrong about whether they understand the intent of a message in the revised format. At some point meeting new requirements and keeping interoperability calls for deprecating and replacing the old format, because treating everything as optional (which some Googlers apparently do, avoiding "required" completely) is almost as bad as having 2^n mostly-untested formats.
- haberman 16y agoThis is the kind of over-engineering analysis that leads to overly complicated systems like XML schema. Just because a field says "optional" doesn't mean it's logically optional. You don't have to make your schema formalism complex enough that it can describe every last rule of what it takes for a message to be valid. In fact you definitely don't want to do that, because it's a horrible amount of complexity in the schema for little gain. Yes, it's true that some Googlers use "optional" instead of "required" everywhere in their .proto files. That doesn't mean that you can omit any field and expect your message to be processed by your peer without error. It just means that you won't get an error at the schema validation level. But the application could still throw an error. More complex rules about what fields must be specified or what values they must have can be described in comments, and enforced with custom validation if necessary. Also, since protobufs support default values, you can define what value will be returned for scalar fields if no value is explicitly sent. This can often be used to define useful default behavior for the case where a field is omitted.
- latch 16y agoan example would be useful
- wccrawford 16y ago"XML deals remarkably well with the full richness of unstructured data." What? No, it doesn't. XML has even more structure than JSON does. Unless you're using it improperly, and the you're negating all the advantages that XML was supposed to bring. It wasn't for 'mixed content'. It was for system interoperability. The advantage that XML brings over JSON is that you can have an external document that you can use to validate any XML you're sending or receiving. You can guarantee it's formatted correctly. JSON provides no such guarantee. Not that that's necessarily a bad thing.
- igrekel 16y agoThink what the author meant is that XML allows documents where the structure is less predefined. It can be the markup that drives the interpretation. With a JSON payload, you need to know how the whole is structured to know how to use it, or build additional mechanisms over it. Also XML brings other useful features like data types.
- Groxx 16y agoIs there a reason JSON fields can't be an object (of sorts) which defines data type, parsing rules, depth to read, circular structures, etc? You can convert, losslessly, between XML and JSON. Doesn't that mean they have identical capabilities, though one may be easier in certain circumstances? There's no "need" in any one which the other doesn't also need.
- c3o 16y ago"You can convert losslessly" – that's not the whole truth. It's an issue that there's no obvious or agreed upon way to render an XML format like ATOM into JSON, because everyone makes up their own transformation rules. Here are two distinct attempts: * http://www.ibm.com/developerworks/library/x-atom2json.html http://www.ibm.com/developerworks/library/x-atom2json.html * http://code.google.com/apis/gdata/docs/json.html http://code.google.com/apis/gdata/docs/json.html
- 16y ago
- andybak 16y agoAnd I'm looking forward to a real world example of 'width' where JSON fails and XML shines.
- DanHulton 16y agoGod, but I hate XML, and sad to say, for the same reason a lot of people hate PHP. I hate XML because it is so mis-used. People try to use it for EVERYTHING and you get these monstrous XML trees with legions of attributes, when what you're trying to send could comfortably expressed in "{success: true, message: 'Record deleted.'}". I admit my ignorance, at least. I'm sure that were I properly shown well-used XML and these wonderful and vast tools that the competent XML worker has at their disposal, I'd be as happy to work with XML as I am to work with PHP. That said, you won't ever catch me trying to write a 3D FPS in PHP, and I hope that if you ever DID catch me doing so, you'd give me the exact same fish-slapping that XML abusers deserve.
- palish 16y agoAt the game studio I work at, they use XML for all scripting tasks: gameplay scripts and UI scripts. As in, XML nodes are "game script commands" which are executed sequentially.
- Udo 16y agoThat's kind of like XSLT, a language with an XML syntax. I can see how that is an appealing use of XML since it basically is a nice representation of a high-level syntax tree. I believe you have come up with a rare example where XML is decidedly better suited than JSON! However, the majority of XML use cases are for document-interchanging applications and the way those are being handled requires massively bloated toolchains of poorly inter-operating software.
- eru 16y agoIf you want a language with a JSON-like syntax, consider using JavaScript. (If you want a language with a uniform syntax, that's easy to parse, choose some form of Scheme or Forth.)
- Udo 16y agoI believe though the point was that I kept wondering what it is specifically that XML does better than JSON, since the article didn't specify. And so it occurred to me that one way to make use of XML's inherent complexity is indeed to represent an instruction tree. But the question here is not what language is best, or even that JSON can used for the same purpose, it was merely the "discovery" of an inherent XML feature that would make reasonable developers prefer it for a certain task. The whole point is moot, however, since XML is actually intended for structured information interchange, regardless of any recent efforts to portray it otherwise, and that's a task which has been thoroughly taken over by JSON in the last few years. > If you want a language with a JSON-like syntax, consider using JavaScript. You're probably right in making this joke though, since JSON is based on JavaScript it certainly owes its popularity mostly to the browser environment.
- DjDarkman 16y agoI find XML to be too heavy and complicated for an interchange format. Quote: "any damn fool could write a better data interchange format than XML"
- prodigal_erik 16y agoThere are hilariously terrible protocol designers out there, and too often nobody with taste manages to stop them. The EDI standards are often cited as examples of how much worse than XML it could get. http://en.wikipedia.org/wiki/EDIFACT#Example http://en.wikipedia.org/wiki/EDIFACT#Example
- patio11 16y agoAfter three years working in Big Freaking Java Web Applications which needed to frequently talk to other Big Freaking Legacy Applications, which is pretty XML's core use case, I struggle to think of a concrete example of a time where XML was a clear win over JSON. I spent a lot more time fighting XML (and, more importantly, libraries built on libraries built on libraries which parsed XML which failed to interact with libraries built on libraries built on libraries which parsed XML in a subtly different and incompatible way) than I spent enjoying the richness of my namespaced mixed content. We were once, no lie, forced into JSON-over-XML because of technical and organizational imperatives that had to be reconciled with the need to ship functional code sometime this year. ("The web services parser is breaking again!? Screw it. We only need three fields. JSON grab bag, serialize as string, read string and interpret as JSON on other side.")
- jimbokun 16y ago"After three years working in Big Freaking Java Web Applications which needed to frequently talk to other Big Freaking Legacy Applications, which is pretty XML's core use case" This is absolutely, positively, NOT XML's core use case. This is yet another case of Big Freaking Java Web Application developers finding a way to add gratuitous complexity, along with their dependency injected, aspect oriented, AbstractDataSourceFactoryFactoryImpls. By the way, a quick Google brought up this, which is evidently not a parody but actual code: http://www.docjar.org/docs/api/org/outerj/pollo/xmleditor/FactoryFactory.html http://www.docjar.org/docs/api/org/outerj/pollo/xmleditor/Fa... The mixed content case does not occur in the vast majority of Big Freaking Java Web Applications.
- patio11 16y agoThere are sometimes when the closest route to the goal, given that you are building a Java web application, runs through very small voice a factory factory. They're a possibly degenerate consequence of the interaction of a few things which, separately, defensible ideas. One of these things is that you want to be able to pass objects over the wire without having to reduplicate all the code on both sides. For example, if you have Student objects in the student management system and also have Student objects in the university SNS, and those systems need to periodically pass data between each other, somebody might say "It would be great if they could do that without us having to custom-write data massaging routines. After all, we already have student code on both systems." and another guy might say "True, true, but we don't want to keep the Student class in sync on both systems neessarily." and then a choice gets made and someone forgets that one of the Students contains a graduation date implemented by a long-forgotten contractor as a wrapper class with some utility functionality which contains a Locale which cannot be serialized and then all of a sudden you are in the sixth layer of XML Hell. Not that I'm arguing with you -- God willing, I will never, ever have to work with Java architecture astronomy again.
- seanalltogether 16y agoI spend all my time doing front-end development and I'm not sure what the author means by "I look forward to seeing what the JSON folks do when they are asked to develop richer APIs". The only deliberately unstructured XML I've worked with is document formats. Where are these rich unstructured APIs that wouldn't otherwise be classified as lazy?
- ilovejson 16y agoI agree XML has it's uses, but I would go for JSON first, XML somewhere after JSON. Any day. I still love JSON, http://ilovejson.com http://ilovejson.com for it's readability and simplicity.
- JonnieCache 16y agoIMO the biggest advantage of JSON on the web is a simple one: fewer bytes on the wire. EDIT: also the concise yet all-encompassing nature of its home at http://www.json.org http://www.json.org
- kenjackson 16y agoIs this really true? It seems like they would be remarkably close in number of bytes on the wire, accept in cases where you generate really poor XML. In fact, in some basic examples I quickly looked at, I think XML actually puts fewer bytes on the wire. I do have to agree with the author though. This feels like a really "meh" discussions.
- xiongchiamiov 16y agoIf the actual data is small compared to the tag names, then XML will be significantly larger.
- JonnieCache 16y agoAlso if the data scheme is very simple and only for consumption by the person who designed it, one can eschew tag names altogether and just have an array and rely of the order of elements to define meaning.
- Goladus 16y agoI have the same reaction as the author, although for perhaps a slightly different reason. XML has far less syntax than JSON, and strings are the default input. So XML is better when you have large chunks of text want to add structure and meaning to abstract subcomponents, and want it to be human-readable. JSON is better when you need to organize lots of little pieces of data. Unfortunately, neither use case is typical. Usually it's a mix of both, and it's not worth using both formats so you have to pick just one.
- sigzero 16y agoThese XML vs JSON talks are tiring. They solve entirely different problem domains. XML is for documents and JSON is for data. It is not the fault of the XML spec that programmers go nutso on its usage outside of its problem domain.
- j_baker 16y agoI don't understand your point. Documents are data, are they not?
- alavrik 16y agoDocuments are semi-structured data, that is they are a mix of text and structured data elements put into a structured hierarchy.
- dfox 16y agoproblem is in that 90% of applications does not need to process mixed-content documents, yet because XML was the new cool-and-interoperable-solution-to-everything it got to be used for anything and mainly for things that are only simple structured data. Probably because you can abuse it for that and as a bonus you get lots of tools (like XSLT)
- grovulent 16y agoI'm no expert on these things... But generally what I want out of a data stream is a set of name/value pairs. That's it. I really don't understand why, but trying to get those name/value pairs from an xml based api is always an adventure. Well - I know why. It's because everyone structures their data differently when they create xml. Sometimes the name I'm after will be a node name. Sometimes it will be a value of a text node. Sometime the value that properly maps to that name will be a textnode 3 layers deep, sometimes it will be an attribute value. I'm still a newb but I've worked with about 8 different apis now. For the json apis I wrote one recursive function that worked on all of them to get the data I wanted... It was about 8 lines of code. For the XML apis, I STILL haven't figured out how to write a recursive function which works for ONE of them, let alone all of them. (while relying on python minidom to do the actual parsing for me).
- plq 16y agoone good property of xml is that it's also a database. it can be queried once inside, say, postgresql's xml type. json lacks that level of support from mainstream relational databases, which makes it a waste of time for certain use-cases.
- simonw 16y agohttp://wiki.postgresql.org/wiki/JSON_datatype_GSoC_2010 http://wiki.postgresql.org/wiki/JSON_datatype_GSoC_2010
- b-man 16y agoHow about SXML? http://okmij.org/ftp/Scheme/SXML.html http://okmij.org/ftp/Scheme/SXML.html
- dstein 16y agoJSON falls flat when trying to define data types and serialize objects. var foo1 = new Foo1(); foo1.bar1 = new Bar1(); var foo2 = new Foo2(); foo2.bar2 = new Bar2(); If you were to serialize that data using XML you might do this: <Foo1 name="foo1"> <Bar1 name="bar1"/> </Foo1> <Foo2 name="foo2"> <Bar2 name="bar2"/> </Foo2> But now what if you use JSON? You'll probably end up with something like this: { "foo1" : { "class" : "Foo1", "@bar1" : {"class":"Bar1"} }, "foo2" : { "class" : "Foo2", "@bar2" : {"class":"Bar2"} } } Generally it makes sense to use XML for imperative data structures, and use JSON for functional data structures.
- j_baker 16y agoI don't see what you're getting at. First of all, how is example 1 better than example 2? And how does that prove that JSON is better for functional data while XML is better for imperative data?
- dstein 16y agoIt boils down to JSON being schemaless. In XML the data type is implicitly defined by the tag, whereas in JSON you need to arbitrarily encode the datatype as a property and somehow differentiate between properties and child elements/objects.
- murrayh 16y agoHow about this? <Foo1 name="foo1"> <Bar1 name="bar1"/> </Foo1> <Foo2 name="foo2"> <Bar2 name="bar2"/> </Foo2> [["Foo1", {"name" : "foo1"}, [ ["Bar1", {"name" : "bar1" }] ], ["Foo2", {"name" : "foo2"}, [ ["Bar2", {"name" : "bar2" }] ]] I personally wouldn't do this, but the arbitrary encodings between the two samples are now equivalent.
- dstein 16y agoNo, I think you proved my point.
- jtchang 16y agoXML just gives me a headache. I just like to point out that there was a time before XML. Fixed width formats are still very common when working with mainframe systems. Surprisingly, they aren't actually that bad to work with.
- chipsy 16y agoI am a huge JSON fan, but I have still managed to find a few cases where I can make use of XML - the mixed-mode situations the author describes. To generalize my viewpoint: XML is a good way to mix data and data processing, or mash up multiple DSLs in polyglot fashion. When stream-parsed, one can imagine great use-cases of XML, where an early node adds context, references and functionality for other data later in the document. But it sucks for plain old standalone-structured data, which is the kind of data we usually like to _store_.
- nradov 16y agoThe lack of a standard for handling namespaces is a fatal flaw in JSON for many applications. Some of the healthcare data exchange formats we deal with require mixing multiple XML namespaces together; it's complex, but it works. Trying to do the same in JSON would be a huge mess. http://blogs.sun.com/bblfish/entry/the_limitations_of_json http://blogs.sun.com/bblfish/entry/the_limitations_of_json
- sielskr 16y agoIn various conversations on the internet, starting with comp.text.sgml where XML was born in the 1990s, I have seen many signs that someone hopes or expects some of the immense richness in meaning that humans assign to words and other symbols to be carried along by the XML. In other words, I've seen many special cases of the mistaken belief that programmers of today can do things that only researchers of the future wielding human-level AI will be able to do -- and that just has to cause grief if those hopes and expectations inform IT decisions.
- famousactress 16y ago<omgThisIsSoHumanReadableICannotBelieveIt>1</omgThisIsSoHumanReadableICannotBelieveIt> Good riddance.
- wvenable 16y agoI remember when XML was still young and I was introduced to XML-RPC. It was truly beautiful protocol -- straight forward, human readable, human writable, and easy to implement. I wrote desktop apps that connected to web services written in different languages on different platforms and it all worked. Then SOAP happened. The same philosophy as XML-RPC gone horribly wrong: a human readable format no longer readable by humans, insanely complex, and hard to implement properly (even with the libraries). Frustrating all around. Although JSON is much simpler than XML, there's no reason why someone couldn't invent something as horrible as SOAP (or XSLT) on top of it. I can only hope that JSON implementers see the value of keeping things simple. Perhaps the existence of XML as an "enterprise" technology will help differentiate JSON in that way.
- roel_v 16y agoI used xmlrpc for this exact purpose (desktop to 'web service' - now there's a buzzword I haven;t heard in 5 years!) early this year, it was awesome. On the server side I had PHP's simple xml api, which also made pretty much all of the pain of parsing xml go away.
- johnm 16y agoFWIW, if you change "unstructured" to "semi-structured" then Norm's argument works much better.
- johnm 16y agoFWIW, if you change "unstructured" to "semi-structured" then Norm's argument works much better.
- est 16y agoI remember a while ago I read a submission on HN (or reddit?) talking that a founder of jabber protocol is ditching XML for something new. Anyone remember?
- joseacta 16y agoI think they shouldn't be mixed. JSON has its strong attributes as also does XML. I started working with XML on 1999 and started doing AJAX style web applications on that time using XML/XSL and Internet Explorer XMLHttpRequest. So now we have JSON, good, excellent addition. Is fast and simple to implement. But to deprecate XML, I don't think so. Imagine just an EDIFACT document in JSON?