3 ms·

"XML is very bad at handling binary data." That is mostly false. No, it is not. Encode a reasonably sized JPEG (say, 500k-1MB) in an XML document. Try measurin

"XML is very bad at handling binary data." That is mostly false. No, it is not. Encode a reasonably sized JPEG (say, 500k-1MB) in an XML document. Try measuring how quickly you can parse that XML document on a lower-end device. Now try encoding a batch of thumbnails (say, 30k each) in an XML document. Now try seeing how fast you can parse 500 of them on a lower-end device. There is no obviously winner (widely adopted) standard for binary format XML although there will be some day. "Some day" doesn't do us any good today.

petewarden 17y ago
Got any control characters (below 0x10 iirc) inside your CDATA block? So sorry, expat and conforming parsers will die. How to strip those characters out of your input files in PHP is still one of my most popular blog posts. XML is horrible for binary data, and the pedantry of parsers makes it worse.
nicpottier 17y ago
XML wasn't really ever made to be high performance though. It was made to be easy to read for humans and relatively easy to generate. At those two things it is awesome. If you are sticking binary data in XML and passing it over the wire to someone you are probably 'doing it wrong'. :) There are obviously much fast formats for that type of thing, but they have their own drawbacks which are significant. Protocol Buffers are crazy fast but aren't self describing or human readable. JSON is really just a different representation of XML on many levels and certainly doesn't deal with binary data any more elegantly. (base64 here we come!) If you have binary data and plenty of CPU on both ends, then gzipping XML gets you pretty close to the native size actually and can be a good solution. You keep the human readability and the compression can be done on the transport layer if it is HTTP. If you don't have much CPU and want easy parsing, it is very easy to write a binary format for XML. I've done this for products that are sending down data to cell phones (including images). The native format is XML, because it is easy to author in templates and easy to debug, but with an extra parameter it gets thrown into a denser and very fast to parse format for the other end to deal with. This costs a bit of CPU on the server, but that generally isn't the problem. Anyways, I would agree that the main author's points are pretty ridiculous. They either point to him not really understanding XML or never really considering the pros and cons of the alternatives. XML is great for what it is, and it has its drawbacks, but as a universal transport it is pretty rad.
- tentonova 17y ago
  There are obviously much fast formats for that type of thing, but they have their own drawbacks which are significant. Protocol Buffers are crazy fast but aren't self describing or human readable. Why does that matter? The fact that they aren't self-describing more or less requires that you use a published message definition, which solves one of the major problems of XML schemas -- nobody actually uses them, and many that do use them incorrectly. XML is great for what it is, and it has its drawbacks, but as a universal transport it is pretty rad. Having wasted far too much of my life on parsing XML documents, I simply can not agree. XML as a universal transport just sucks compared to the modern alternatives.
  - nicpottier 17y ago
    Are you pitting XML against Protocol Buffers? Because they are completely different things. Being self describing is a big deal.. it makes interchange far far easier and obvious. As does being human readable, debugging a web API in a browser or via curl is a serious plus. Having just spent the past two days reverse engineering an API that used protocol buffers, I can tell you I would have much rather done it against XML. :) As for parsing, I guess I don't get what the big deal is. I've long since given up on SAX parsers and occasionally use pull parsers, but for the majority of applications using a DOM parser gives you a very OO friendly approach to turning that XML into your objects, and it is dead simple. Sine the generated Protocol Buffer objects are immutable and can't be used as base classes, I suspect the amount of code to turn a 'message' to your domain objects and vice versa is very similar between protocol buffers and a DOM impl. At least it has been in my experience, neither is hard.
    - prodigal_erik 17y ago
      I've never seen value in self-describing or human readable messages. You need some kind of relatively unique identifier for the format of a given message and any optional payload, but beyond that, if a receiver doesn't have any code that knows what to do with a given message, what's the point of being able to deserialize it? And I think debugging an API without making tools is a waste of effort, for the same reason that you use curl rather than typing "POST..." into netcat.
dasht 17y ago
XML is an extensible framework starting with two main things: a data model (very good) and a text-based surface syntax that is portable across a great many character set encodings. Encoding binary data (as, say, CDATA) and then shipping or storing it in the text-based syntax is every bit the lose you suggest it is. Binary XML formats, however, can answer your objection - the main problem being that no good binary XML format is yet widely adopted. You say "`Some day' doesn't do us any good today." I half agree. It is true that, today, if you want to use XML for large binary data you have to place your bet on a suitable binary format and you face a rocky road. XML "sucks" in that sense. On the other hand, if you can possibly - for the problem you are trying to solve - make a good bet on a binary format for XML, then it is a fine choice because the data model for XML as interchange format is excellent. So using it for binary today is not an option in many cases and has an element of risk on other cases - but longer term is a winning bet. Your benefit for considering it today is that longer term pay-off. "Go long" on XML.
- tentonova 17y ago
  XML proponents always trot out "we're just not there yet" or "the basic idea is good, we just need some small fixes", or "parsers just need improvement -- use a better parser library", or any other number of excuses as to why XML doesn't actually suck, it's just a victim of poor implementations or to-be-fixed design inadequacies. Meanwhile, Protobuf and Thrift work today. They're fast, they're size efficient, and unlike XML, they're simple. It's time to stop making excuses for XML. It's an inexcusable ball of poorly designed unjustifiable complexity created by architecture astronauts run amok. I'm disappointed at the upvotes on a whole page of platitudes on why XML isn't a bad format, it's just misunderstood.

petewarden 17y ago

Got any control characters (below 0x10 iirc) inside your CDATA block? So sorry, expat and conforming parsers will die. How to strip those characters out of your input files in PHP is still one of my most popular blog posts. XML is horrible for binary data, and the pedantry of parsers makes it worse.

nicpottier 17y ago

XML wasn't really ever made to be high performance though. It was made to be easy to read for humans and relatively easy to generate. At those two things it is awesome. If you are sticking binary data in XML and passing it over the wire to someone you are probably 'doing it wrong'. :) There are obviously much fast formats for that type of thing, but they have their own drawbacks which are significant. Protocol Buffers are crazy fast but aren't self describing or human readable. JSON is really just a different representation of XML on many levels and certainly doesn't deal with binary data any more elegantly. (base64 here we come!) If you have binary data and plenty of CPU on both ends, then gzipping XML gets you pretty close to the native size actually and can be a good solution. You keep the human readability and the compression can be done on the transport layer if it is HTTP. If you don't have much CPU and want easy parsing, it is very easy to write a binary format for XML. I've done this for products that are sending down data to cell phones (including images). The native format is XML, because it is easy to author in templates and easy to debug, but with an extra parameter it gets thrown into a denser and very fast to parse format for the other end to deal with. This costs a bit of CPU on the server, but that generally isn't the problem. Anyways, I would agree that the main author's points are pretty ridiculous. They either point to him not really understanding XML or never really considering the pros and cons of the alternatives. XML is great for what it is, and it has its drawbacks, but as a universal transport it is pretty rad.

tentonova 17y ago
There are obviously much fast formats for that type of thing, but they have their own drawbacks which are significant. Protocol Buffers are crazy fast but aren't self describing or human readable. Why does that matter? The fact that they aren't self-describing more or less requires that you use a published message definition, which solves one of the major problems of XML schemas -- nobody actually uses them, and many that do use them incorrectly. XML is great for what it is, and it has its drawbacks, but as a universal transport it is pretty rad. Having wasted far too much of my life on parsing XML documents, I simply can not agree. XML as a universal transport just sucks compared to the modern alternatives.
- nicpottier 17y ago
  Are you pitting XML against Protocol Buffers? Because they are completely different things. Being self describing is a big deal.. it makes interchange far far easier and obvious. As does being human readable, debugging a web API in a browser or via curl is a serious plus. Having just spent the past two days reverse engineering an API that used protocol buffers, I can tell you I would have much rather done it against XML. :) As for parsing, I guess I don't get what the big deal is. I've long since given up on SAX parsers and occasionally use pull parsers, but for the majority of applications using a DOM parser gives you a very OO friendly approach to turning that XML into your objects, and it is dead simple. Sine the generated Protocol Buffer objects are immutable and can't be used as base classes, I suspect the amount of code to turn a 'message' to your domain objects and vice versa is very similar between protocol buffers and a DOM impl. At least it has been in my experience, neither is hard.
  - prodigal_erik 17y ago
    I've never seen value in self-describing or human readable messages. You need some kind of relatively unique identifier for the format of a given message and any optional payload, but beyond that, if a receiver doesn't have any code that knows what to do with a given message, what's the point of being able to deserialize it? And I think debugging an API without making tools is a waste of effort, for the same reason that you use curl rather than typing "POST..." into netcat.

dasht 17y ago

XML is an extensible framework starting with two main things: a data model (very good) and a text-based surface syntax that is portable across a great many character set encodings. Encoding binary data (as, say, CDATA) and then shipping or storing it in the text-based syntax is every bit the lose you suggest it is. Binary XML formats, however, can answer your objection - the main problem being that no good binary XML format is yet widely adopted. You say "`Some day' doesn't do us any good today." I half agree. It is true that, today, if you want to use XML for large binary data you have to place your bet on a suitable binary format and you face a rocky road. XML "sucks" in that sense. On the other hand, if you can possibly - for the problem you are trying to solve - make a good bet on a binary format for XML, then it is a fine choice because the data model for XML as interchange format is excellent. So using it for binary today is not an option in many cases and has an element of risk on other cases - but longer term is a winning bet. Your benefit for considering it today is that longer term pay-off. "Go long" on XML.

tentonova 17y ago
XML proponents always trot out "we're just not there yet" or "the basic idea is good, we just need some small fixes", or "parsers just need improvement -- use a better parser library", or any other number of excuses as to why XML doesn't actually suck, it's just a victim of poor implementations or to-be-fixed design inadequacies. Meanwhile, Protobuf and Thrift work today. They're fast, they're size efficient, and unlike XML, they're simple. It's time to stop making excuses for XML. It's an inexcusable ball of poorly designed unjustifiable complexity created by architecture astronauts run amok. I'm disappointed at the upvotes on a whole page of platitudes on why XML isn't a bad format, it's just misunderstood.