27 ms·

XML is almost always misused

201 points by mrzool 7y ago

sosuke 7y ago
The larger your XML file is the more accurately you're using it. Less " and more <>. I made these mistakes, using XML like a I was writing an HTML doc.
mpweiher 7y ago
Hmm...somewhat disagree with the "correct" way to express a dictionary. I prefer: <root> <Name>John</Name> <City>London</City> </root> Removes one level of indirection, XML already has keys.
- progval 7y ago
  That's not a good way to express a dictionary because it does not allow arbitrary strings as key names. It's also not a good example of XML, because XML schemas should have a fixed list of tag names.
  - mpweiher 7y ago
    XML allows arbitrary strings as key names just fine.
    - progval 7y ago
      No, you can't have quotes, spaces, etc., because with GGP's scheme it would be a tag name, and these are invalid in tag names.
  - sorenjan 7y ago
    But the starting point in the example was <item name="name" value="John" /> <item name="city" value="London" /> where the key names are used as attributes, so it wouldn't work with arbitrary key names either, right?
    - progval 7y ago
      It would work (kind of); most XML parsers/generators would take care of escaping and unescaping quotes; but there's no way in the XML spec to escape characters in tag names.
- Felk 7y ago
  Which works if you either expect a dictionary with a predefined set of fields (which isn't really a dictionary then), or parse your xml in a way that handles arbitrary tags. For a generic dictionary the shown approach is still the way to go, if you really want to use XML for that.
  - mpweiher 7y ago
    XML parsers generally allow arbitrary tags.
- srott 7y ago
  you've just invented plist, wonder if it can be validated by a schema.. <dict> <key>CFBundlePackageType</key> <string>FMWK</string> <key>CFBundleShortVersionString</key> <string>4.8</string> ...
  - progval 7y ago
    No, GP didn't. They use tag names as keys. But you answer your question, yes this format can be expressed with XML-Schema: <xsd:element name="dict"> <xsd:complexType> <xsd:sequence minOccurs="0" maxOccurs="unbounded"> <xsd:element name="key" type="xsd:string" /> <xsd:choice> <xsd:element name="string" type="xsd:string" /> <xsd:element name="integer" type="xsd:integer" />  </xsd:choice> </xsd:sequence> </xsd:complexType> </xsd:element>
    - spiralx 7y ago
      Or in RELAX-NG compact :) grammar { start = Dict Dict = element dict { DictItem* } DictItem = element key { text }, (element string { text } | element number { xsd:integer }) }
- jimktrains2 7y ago
  The main issue I see with that is that if it's a true dictionary, then those elements will constantly be different, which is weird. Now, if we're just encoding a dictionary that's an already an encoding of an object, then yeah, let's just encode the object directly like you are above.
- Sharlin 7y ago
  This is not a dictionary, it’s a record. Unfortunately this fairly fundamental distinction has been thoroughly muddied by certain languages that want to use associative arrays for everything.
  - mpweiher 7y ago
    Hmm...you seem to be mapping programming language concepts 1:1 to XML. That is a mistake.
    - Sharlin 7y ago
      I'm not the one doing that. People using XML as something else than a document markup language are. Which is what the article author is complaining about. But if you really want to do that, then at least do it properly. Record fields have predefined names, just like XML elements have predefined names. Dictionary keys can be arbitrary, like XML text nodes or attribute values but emphatically not element or attribute names, unless your "XML" is actually just tag soup.
- deleted 7y ago
  [deleted]
- benibela 7y ago
  That is definitely the best to use with xpath
foolfoolz 7y ago
turns out a well specified format that has a lot of parsers available is useful for more than just a markup language. xml is great at data formatting, a little more verbose than alternatives but also a lot more feature rich
- kazinator 7y ago
  The format wasn't well-specified and didn't have a lot of parsers in 1996 though. The parsers came after the decision was made by a lot of people to use a markup language inappropriately for structured data.
CobrastanJorji 7y ago
> Here are some very frequently occurring examples of bad schema design: (4 lines, 75 characters) > Here's the right way: (10 lines, 133 characters) I have a suspicion as to what went wrong.
- SamBam 7y ago
  > "But if the people who made the strange decision to use XML as a data format [...] they might realise that what they're doing is unsuited to it and unergonomic The author's point is that XML should not be a data format.
  - zwkrt 7y ago
    Is there logic to the authors assertion past “that isn’t what XML was intended for”? It is a pretty nice data format if you want wide compatibility and schema integration.
    - mrzool 7y ago
      The point is, XML is not a data format. It’s a markup language.
      - LudwigNagasena 7y ago
        I just invented XDF, it is exactly like XML, but it can be used to represent arbitrary data structures. Do you have any objections now?
        benibela 7y ago
        That is only valid if the XDF standard has been published by the W3C
        gnulinux 7y ago
        This doesn't make any sense. W3C doesn't restrict/define use cases of XML, it defines structure and semantics thereof; using it as-if it is an XDF document is perfectly ok, just like you can use XML data for your DASH stream etc... It's a structure on top of XML structure.
theamk 7y ago
One thing this misses in the "dictionary" example is that tools (like xpath) push you towards "key in attribute" selection. One of the most common operations we do with dictionaries is lookup by known key, and storing the key in attributes makes it much easier.
- spiralx 7y ago
  "Key in attribute" is the correct way to do it, it's just that his examples are absolutely terrible and make no sense at all. A completely unstructured list of key-value pairs is overkill for any structured data format.
deleted 7y ago
[deleted]
zippergz 7y ago
If a piece of technology is "almost always" misused, is that the fault of the users, or the technology?
- dimitar 7y ago
  Exactly, this is some variant of No true Scotsman
- deleted 7y ago
  [deleted]
- scarygliders 7y ago
  If you mean "users" as in "users of the XML format", then it's the fault of developers as 'users' of their applications don't have any choice in the matter.
  - zippergz 7y ago
    I mean "users" exactly the same way "used" was meant in the author's statement. So yes, the developers, who are the people who are using XML.
- Gibbon1 7y ago
  Marketing usually.
deleted 7y ago
[deleted]
deleted 7y ago
[deleted]
alkonaut 7y ago
I’m going to just argue the exact opposite of the article: xml and json are both structured data formats useful for tree like data graphs, such as objects. Whether that was the intended purpose when xml was designed is irrelevant. It’s what xml is used for in almost every case. The author also doesn’t suggest what should be used instead to encode structured data, or perhaps more importantly what should have been used to encode graph like things such as map/lists/objects in the 2000’s. Json really hasn’t been an alternative until quite recently (10 years ago?). In fact reading the article carefully I fail to see the author argue why xml shouldn’t be used as a data format either.
- miggol 7y ago
  I'd argue that for data storage purposes, you'd like to have a low metadata to information ratio. In the examples the author gives this seems to be the main problem, with way more characters being used for markup than for content. Compare that to JSON or TOML, which are more human-friendly and waste fewer bytes on structure to convey the same information. When used for data storage, two XML files of the same schema describing two completely different objects are likely to share a large amount of content, which is wasteful and gets in the way.
  - RantyDave 7y ago
    I beg to differ. For a start, XML compresses well and besides, storage is monster cheap these days. XML is a better storage format because it documents what the data is (a title, a reference etc) as well as the data itself. There are many better reasons to hate XML.
    - hnick 7y ago
      XML does compress well as text or over the wire but the parsing trees can be quite large in memory and processing consumption. At least in Perl I've had enough scripts crash out due to this overhead when implementing the common/naive solution using off the shelf modules. You can get around this by choosing between DOM or SAX but I consider that a symptom of the problem, you choose XML to solve a problem and now you have another problem to solve.
altmind 7y ago
Can I have my gripe with apple plists? <key>CFBundleDisplayName</key> <string>TextEdit</string> <key>NSHumanReadableCopyright</key> <string>Copyright 2019</string> This may be perfectly parsable by a SAX parser storing some state, but its totally not processable by xslt.
- k__ 7y ago
  How old is this format? I had the impression mobile was some new try at frontend tech, but somehow iOS and Android threw a whole bunch of outdated stuff at me. I mean, before the iPhone I designed a XML based ETL config system and tried to avoid all the common XML errors, then I start doing a mobile app 10 years later and it's like all that knowledge was forgotten...
- mikl 7y ago
  Yeah, that's awful XML. Ironically, they used to have a decent readable format in NeXTStep, it was only later it was XML-ified.
- nwellnhof 7y ago
  > but its totally not processable by xslt. It's a bit inconvenient but perfectly processable with XPath: /dict/key[.='CFBundleDisplayName']/following-sibling::string[1]
- spiralx 7y ago
  I remember when Neverwinter Nights 2 came out touting that all of its data files were in XML for ease of modifying in user extensions. So I had a look and was it XML? Was it fuck, it was like a GeoCities novice's idea of how to code HTML - absolutely unparseable in any way unless you're the idiot who decided what the codebase needed was another shitty homegrown parser. And yes, plists suck and make your XPath selectors ugly, although you could write a function to abstract them out.
mojuba 7y ago
That's a very useful and sobering view actually. Unfortunately for the XML format it wasn't designed to prevent its own abuse. But anyway, XML is for documents sounds like a good and acceptable paradigm. That being said, one subtle and important (and often overlooked) difference between XML and, say JSON is that you can stream XML while parsing it on the application level, whereas JSON can not be parsed by the application due to arbitrary ordering of keys. (Of course lower level parsers use streaming anyway, but that's not the point) In fact you not only can but you should parse XML while streaming it. This is another common abuse: wherever you look you see some high level function that loads an entire parsed XML structure into memory at once. But once you start asking yourself where the file may be coming from you realize that your system may be open to denial of service attacks. E.g. is your system ready to receive a 16GB XML file?
- Bouncingsoul1 7y ago
  Exactly this, last time I checked the dump of german wikipedia was around 16GB so you'll find these files in the wild
- fanf2 7y ago
  There is no reason you can’t do a SAX-style parser for JSON. A quick bit of searchengineering will find dozens, eg. https://github.com/dgraham/json-stream https://github.com/dgraham/json-stream
  - mojuba 7y ago
    You can in principle, but because the order is not guaranteed, you may find yourself accumulating things in memory. It depends on the task of course, but neither standard parsers nor generators are required to send adjacent JSON entities in a specific order. I remember someone describing a trick where instead of sending a JSON array they'd just send a stream of JOSN objects, one per line, so that the receiving end could parse the data in a streaming fashion. But that's not JSON anymore.
    - hnick 7y ago
miggol 7y ago
The worst example of this that I deal with regularly has to be the libvirt domain XML. https://libvirt.org/formatdomain.html https://libvirt.org/formatdomain.html It does occasionally put information outside of the tags, but because there's no logic to when, it's nearly worse.
tehjoker 7y ago
It is difficult for me to see what the real issue is with examples given. It seems to be more an aesthetic preference of the author rather than a technical argument. People can use formats for whatever they want. :P If you told me that the transmission and parsing rate is too slow for their application, that's a real dig at it.
- mehrdadn 7y ago
  I think it's a bit like trying to explain why a Python REPL isn't a substitute for a calculator? Like it can of course do what you want, and you can't "see" what's wrong if you just take what you see literally (you'll obviously get the same answers regardless of what tool you use), but it's just... not meant for that.
kbenson 7y ago
So, what is XML good for? If it's not good for data as everyone says (and I'm not inclined to argue), but it is good for documents, what kind of documents are we referring to? A defined metadata on a text document? A template used with data to generate something else? Is a configuration file a document or data? Where would I want to use XML that something like JSON, a text document, or some combination thereof wouldn't be better? I'm not being facetious, this is an honest question. Where are the "right" places to use XML?
- jerf 7y ago
  I wrote this four years ago: https://news.ycombinator.com/item?id=11446984 https://news.ycombinator.com/item?id=11446984 There are points of disagreement between me and the author, although I wouldn't get too passionate about them. Super-short version, reading over it again, is that XML is very good at what it does, but it really ought to be seen as a relatively specialized data format. It's really good at certain tasks, best-of-breed for a couple of them, and degrades rapidly as you get away from that. JSON is a fairly cheap & fast general-purpose format that's OK at a lot of things, isn't necessarily great at much, but as you get into more specialized use cases, also tends to degrade. Being a general-purpose format, perhaps arguably it degrades more "slowly", but it does degrade. Properly understood, IMHO, their use cases don't overlap much if at all, and the combination of them may cover a lot of space, but are still far, far from the only serialization formats you'll ever need.
- TallGuyShort 7y ago
  Think of semi-structured documents, where you have a list of pre-defined sections. I've seen it in use by insurance companies for case reports, in real estate for appraisals. And of course we've all seen it work well in the form of HTML. There's some structure to all of these examples, but mostly for annotating text sections. You need some flexibility built into the schema to add fields as needed, but you're not dealing with various map / list / primitive data types as a matter of routine. Just making this one up, but if LaTeX wasn't already the standard, I'd also use it if I was digitizing the content academic papers, for instance. You have an a header with metadata, abstract, the body, citations. There's some structure, a need to add some metadata, perhaps flexibly over time, but mostly it's just a document.
benibela 7y ago
The worst XML use I have ever seen are lists generated by Lazarus. Every list. For example in the project files you have: <RequiredPackages Count="5"> <Item1> <PackageName Value="LazUtils"/> </Item1> <Item2> <PackageName Value="treelistviewpackage"/> </Item2> <Item3> <PackageName Value="internettools"/> </Item3> <Item4> <PackageName Value="LCLBase"/> <MinVersion Major="1" Release="1" Valid="True"/> </Item4> <Item5> <PackageName Value="LCL"/> </Item5> </RequiredPackages>
- sedachv 7y ago
  Compare that to SGML: https://leancrew.com/all-this/2014/09/sgml-nostalgia/ https://leancrew.com/all-this/2014/09/sgml-nostalgia/ SGML really needs a revival.
- hyperman1 7y ago
  I once got a really nice one: Software v1.0: Config is a binary blob and everybody curses the nasty editor provided by vendors. Software v2.0: Config is in XML. Thank god! Oh wait. <Binaryblob><Byte value="65"/><Byte value="99"/> ....
robofanatic 7y ago
I guess the correct answer depends upon the requirement. I like this way <root> <item key="name">John</item> <item key="city">London</item> </root> So I can use this xpath to get the person's name: //root/item[@key="name"]/text() Not sure what would be the xpath to get the name if the XML was <root> <item> <key>Name</key> <value>John</value> </item> <item> <key>City</key> <value>London</value> </item> </root> This is a better example: <employees> <employee id="1"> <field name="name">John</field> <field name="city">London</field> </employee> <employee id="2"> <field name="name">Jack</field> <field name="city">Boston</field> </employee> <employees>
- scandox 7y ago
  If you accept the author's premise then all of these (including his) are wrong uses of XML.
- elFarto 7y ago
  The XPath for the second one would be: //root/item/key[text()="Name"]/../value/text()
  - icebraining 7y ago
    Alternatively //root/item[key[text()="Name"]]/value/text()
    - benibela 7y ago
      You usually do not need to use text() in xpath, so this should work the same: //root/item[key="name"]/value
- commandlinefan 7y ago
  Well, actually, an even better example is: <employees> <employee id="1"> <name>John</name> <city>London</city> </employee> <employee id="1"> <name>Jack</name> <city>Boston</city> </employee> I know what you’re trying to do there - you’re trying to “future-proof” your schema by allowing introduction of arbitrary new elements. Which means that there’s no standard way to guard against somebody omitting a required field (like “name”) or adding a new field like “creditCardNumber” - other than to document your acceptable key values in a non-standard format and add defensive code that a validating parser would have given you. You’re better off taking as much advantage of the format as you can.
bullen 7y ago
I would love to know what people think of my XML node-graph/tree editor I made before JSON became mainstream (my excuse): http://rupy.se/logic.jar http://rupy.se/logic.jar It basically names the tag what you name the node. :S - You link/unlink nodes (I called them entities! Xo) by right-click-dragging between them. - You copy stuff by right-click-dragging to an empty space. - You delete by grabbing something by left-click-holding and pressing the delete key. - Oh, and nodes are completely tree structure expandable, just drag-drop attributes on nodes and nodes inside nodes. The editor uses lightweight rendering so you can have a ton of elements with good performance. (I know, not super intuitive; but very handy once you know about these.)
davidw 7y ago
I love the quote I originally saw on the Nokogiri (Ruby XML lib) site: "XML is like violence – if it doesn’t solve your problems, you are not using enough of it."
scarygliders 7y ago
I have always thought that using XML as a format for storing and retrieving configuration files, was complete insanity. Which is why I still use the simple, effective, INI format for configuration files for applications I write. XML for config files is madness personified.
deanCommie 7y ago
I think that Software Engineers should take influence from Authors (after all, are we not all craftsmen/artisans?) and incorporate the philosophies from https://en.wikipedia.org/wiki/The_Death_of_the_Author https://en.wikipedia.org/wiki/The_Death_of_the_Author The idea, for those not familiar, is that once a work of art is published (a novel, a poem, a song, a painting), it speaks for itself, and authorial intent no longer matters. That is, meaning and purpose are in the eye of the beholder/consumer. And there is no right or wrong way to "interpret" art. If someone finds meaning that the author did not intend, it is just as valid as a deeply hidden but intentional allegory they intentionally placed in when they were writing. The relevance to software is it applies to APIs, specifications, standards and formats. There is no such thing as users using your software or specification "wrong" - if they insist on doing so, the meaning has evolved. Evolve with it or die.
- rkagerer 7y ago
  That's a little extreme but you raise a good point. I think a talented spec designer anticipates how their work might be interpreted / used / abused, and, like an adroit villain, nudges their audience toward tenets their grand scheme seeks to achieve.
hashberry 7y ago
> a simple test for determining if an XML schema is well designed: remove all tags and attributes from it ... If what you have left over does not make sense ... you shouldn't be using XML at all. Magento 2 (acquired by Adobe for $1.68bn) uses XML to render its layouts. Here's some fun XML for the checkout page: https://github.com/magento/magento2/blob/2.3-develop/app/code/Magento/Checkout/view/frontend/layout/checkout_index_index.xml https://github.com/magento/magento2/blob/2.3-develop/app/cod...
unilynx 7y ago
XML is a actually wonderful format for data and especially extensible configuration if you combine it with XML Schema and CSS selectors... - XML schemas give you a ready-to-use format to describe, restrict and document available configuration settings. The unique keys help and a libxml2 gives ready to use validation, even if you may need to 'translate' its error messages before showing them to end users - XML schemas also support other annotations so you can further generalise your configuration readers by recording the necessary bindings in the XML schema itself, allowing to use it eg to define application user interfaces. - Almost any text editor can do basic syntax validation preventing most typographic errors, and even better if they can read the schema - XML schemas are extensible using <import>s, but namespaces still enforce some separation. You can define explicit points where plugins extend your configuration format using <any> - Human editable - closing tags are noisy but more readable than }],{}] when non-programmers may have to edit these files just to add a few extra textfields to an UI. - Better datatype support, eg datetimes, by using XML schemas. JSON's type support is too limited - Support for comments! - And once you've verified the schema... CSS selectors and DOM APIs to actually process the XML documents. YAML fixed quite a few things, but still no date times or as far as I know standardised approaches to defining schemas. And I've lost count at how any attempts exist to add schema information or namespacing to JSON... But for markup... we may be better off to just use markdown inside CDATA blocks
- sedachv 7y ago
  > XML schemas give you a ready-to-use format to describe, restrict and document available configuration settings. As someone that likes and uses s-expressions, I never thought I would find myself defending XML, but here we are in 2019, no one understands basic parsing theory anymore, and file formats have "evolved" to hot garbage like YAML and TOML. XML has some great tools in comparison: http://xmlsoft.org/xmllint.html http://xmlsoft.org/xmllint.html https://relaxng.org/ https://relaxng.org/ > But for markup... we may be better off to just use markdown inside CDATA blocks SGML can still make a comeback: https://leancrew.com/all-this/2014/09/sgml-nostalgia/ https://leancrew.com/all-this/2014/09/sgml-nostalgia/
  - novok 7y ago
mickduprez 7y ago
XML is/can be much more than a markup language and yes, it can be used very badly but this is usually by inexperienced 'data wranglers' who don't understand the difference between data attributes and data proper. While XML can seem cumbersome (compared to JSON say) it is a very good 'data transport' tool when used correctly with a sensible schema (XSD). For example, we use XML as a 'vendor neutral' data format to export/import CAD geometry and associated data for town utilities such as buildings, pipes, roads etc. All this data has to be validated against the schema to ensure its correctness. Using a schema like this enables the city council to import this XML into the GIS system to be used for asset management, financial planning etc. A good schema can be key to sharing XML effectively between departments/applications and being a markup language this data can also be viewed independently using XLST.
h2odragon 7y ago
I present, in the spirit of 'worst XML ever', the docs for ScriptXML: https://www.egosoft.com:8444/confluence/display/XRWIKI/Mission+Director+Guide https://www.egosoft.com:8444/confluence/display/XRWIKI/Missi... This is the language used in the game X4 Foundations (and others in the series). An example of its use (mine, i claim no grace in it): https://github.com/h2odragon/dragoncommands/blob/master/aiscripts/deployglobe.xml https://github.com/h2odragon/dragoncommands/blob/master/aisc... ... XML is not a great format for an extension language, I have to say.
- Gibbon1 7y ago
  Eclipse project files are also hot garbage.
LameRubberDucky 7y ago
For those wondering what you do with XML as a document markup language, see the XML document that is the specification for XML. I had to look at the page source to determine it really is an XML document. Looks like an HTML document. https://www.w3.org/TR/xml/REC-xml-20081126.xml https://www.w3.org/TR/xml/REC-xml-20081126.xml
- yellowapple 7y ago
  Jesus. That's slick. Almost makes me appreciate all the bells and whistles of XML. Almost.
- spiralx 7y ago
  I've got a stylesheet I wrote that turns HTML or XHTML into a fully indented and highlighted representation of itself which I was quite proud of :) I used to spend a lot of time writing XSLT and XQuery lol.
commandlinefan 7y ago
The first rule of XML is: whatever you're doing with it, that's not what it was for.
micimize 7y ago
I can't find much to corroborate this article's take. RDF is a stark counter-example - a standard from the W3C. It has endorsement from Tim Bray, one of XML's co-authors.
tannhaeuser 7y ago
> In 1996, XML was invented. XML wasn't an original invention; it is specified as a proper SGML subset. From the XML spec: > The Extensible Markup Language (XML) is a subset of SGML that is completely described in this document. Now I totally agree that SGML and XML aren't for service payloads and config files. The sole purpose of markup languages is representing structured text. And arguably, SGML fills this role much more adequately than XML today as it can represent (via the SHORTREF mechanism) custom Wiki syntaxes such as markdown and others, and in contrast to XML, can deal with the largest corpus of markup out there eg. can parse HTML with all its minimization features such a omitted tags, enumerated and unquoted attributes, etc. See [1] for a practical introduction (disclaimer: link to a tutorial I held last month at ACM DocEng). [1]: http://sgmljs.net/docs/sgml-html-tutorial.html http://sgmljs.net/docs/sgml-html-tutorial.html
- catalogia 7y ago
  SGML's implied close tags are a pain in the ass though. End tags in XML are overly verbose, but at least they're required.
  - tannhaeuser 7y ago
    You control whether an element requires start- and end-element tags in your element declaration via "O" (letter O as in "omissible") in the respective tag omission indicator position: <!ELEMENT e - -(f,g,h) -- no tag omission --> <!ELEMENT f O - (#PCDATA) -- start-tag omission --> <!ELEMENT g - O (#PCDATA) -- end-tag omission --> <!ELEMENT h O O (#PCDATA) -- both start- and end- tag omission allowed--> What's painful about end-element tag omission?
  - gnuarch 7y ago
    Can't find the source, but vaguely remember this line from a Usenet or mailing list post, maybe by Tim Bray: The lack of </> in XML is a crime against humanity.
just_myles 7y ago
I agree with this portion 100% The correct way to express a dictionary in XML is something like this: <root> <item> <key>Name</key> <value>John</value> </item> <item> <key>City</key> <value>London</value> </item> </root> In the past I used to create scripts that exported xml from relational data but didn't really understand the right way to build and structure them.
billsix 7y ago
RIP Eric Naggum http://www.schnada.de/grapt/eriknaggum-xmlrant.html http://www.schnada.de/grapt/eriknaggum-xmlrant.html
Quarrelsome 7y ago
I once had to write a data layer in xml, in-situ with a lifespan for up to hours, as more data was appended to it. An invalid xml document that you couldn't load in many xml apis for 99.9% of its lifespan. I begged and pleaded the lead architect to use an sqlite db for the elements of the data until the transaction was complete and then merely produce the xml file at the end, but no. It had to survive power outs too.
lmilcin 7y ago
And JavaScript was never meant to be used to build applications... There are many, many more inventions that are used for different purposes they were meant for. The Internet was created so that US can withstand nuclear attack and it was never meant to be primarily used to spread advertisements. Get over it.
billpg 7y ago
XML with only tags and attributes (nothing but ignored spaces between '>' and '<') is a reasonable structured data format.

201 points by mrzool 7y ago

The larger your XML file is the more accurately you're using it. Less " and more <>. I made these mistakes, using XML like a I was writing an HTML doc.

mpweiher 7y ago

Hmm...somewhat disagree with the "correct" way to express a dictionary. I prefer: <root> <Name>John</Name> <City>London</City> </root> Removes one level of indirection, XML already has keys.

progval 7y ago
That's not a good way to express a dictionary because it does not allow arbitrary strings as key names. It's also not a good example of XML, because XML schemas should have a fixed list of tag names.
- mpweiher 7y ago
  XML allows arbitrary strings as key names just fine.
  - progval 7y ago
    No, you can't have quotes, spaces, etc., because with GGP's scheme it would be a tag name, and these are invalid in tag names.
- sorenjan 7y ago
  But the starting point in the example was <item name="name" value="John" /> <item name="city" value="London" /> where the key names are used as attributes, so it wouldn't work with arbitrary key names either, right?
  - progval 7y ago
    It would work (kind of); most XML parsers/generators would take care of escaping and unescaping quotes; but there's no way in the XML spec to escape characters in tag names.
Felk 7y ago
Which works if you either expect a dictionary with a predefined set of fields (which isn't really a dictionary then), or parse your xml in a way that handles arbitrary tags. For a generic dictionary the shown approach is still the way to go, if you really want to use XML for that.
- mpweiher 7y ago
  XML parsers generally allow arbitrary tags.
srott 7y ago
you've just invented plist, wonder if it can be validated by a schema.. <dict> <key>CFBundlePackageType</key> <string>FMWK</string> <key>CFBundleShortVersionString</key> <string>4.8</string> ...
- progval 7y ago
  No, GP didn't. They use tag names as keys. But you answer your question, yes this format can be expressed with XML-Schema: <xsd:element name="dict"> <xsd:complexType> <xsd:sequence minOccurs="0" maxOccurs="unbounded"> <xsd:element name="key" type="xsd:string" /> <xsd:choice> <xsd:element name="string" type="xsd:string" /> <xsd:element name="integer" type="xsd:integer" />  </xsd:choice> </xsd:sequence> </xsd:complexType> </xsd:element>
  - spiralx 7y ago
    Or in RELAX-NG compact :) grammar { start = Dict Dict = element dict { DictItem* } DictItem = element key { text }, (element string { text } | element number { xsd:integer }) }
jimktrains2 7y ago
The main issue I see with that is that if it's a true dictionary, then those elements will constantly be different, which is weird. Now, if we're just encoding a dictionary that's an already an encoding of an object, then yeah, let's just encode the object directly like you are above.
Sharlin 7y ago
This is not a dictionary, it’s a record. Unfortunately this fairly fundamental distinction has been thoroughly muddied by certain languages that want to use associative arrays for everything.
- mpweiher 7y ago
  Hmm...you seem to be mapping programming language concepts 1:1 to XML. That is a mistake.
  - Sharlin 7y ago
    I'm not the one doing that. People using XML as something else than a document markup language are. Which is what the article author is complaining about. But if you really want to do that, then at least do it properly. Record fields have predefined names, just like XML elements have predefined names. Dictionary keys can be arbitrary, like XML text nodes or attribute values but emphatically not element or attribute names, unless your "XML" is actually just tag soup.
deleted 7y ago
[deleted]
benibela 7y ago
That is definitely the best to use with xpath

foolfoolz 7y ago

turns out a well specified format that has a lot of parsers available is useful for more than just a markup language. xml is great at data formatting, a little more verbose than alternatives but also a lot more feature rich

kazinator 7y ago
The format wasn't well-specified and didn't have a lot of parsers in 1996 though. The parsers came after the decision was made by a lot of people to use a markup language inappropriately for structured data.

CobrastanJorji 7y ago

> Here are some very frequently occurring examples of bad schema design: (4 lines, 75 characters) > Here's the right way: (10 lines, 133 characters) I have a suspicion as to what went wrong.

SamBam 7y ago
> "But if the people who made the strange decision to use XML as a data format [...] they might realise that what they're doing is unsuited to it and unergonomic The author's point is that XML should not be a data format.
- zwkrt 7y ago
  Is there logic to the authors assertion past “that isn’t what XML was intended for”? It is a pretty nice data format if you want wide compatibility and schema integration.
  - mrzool 7y ago
    The point is, XML is not a data format. It’s a markup language.
    - LudwigNagasena 7y ago
      I just invented XDF, it is exactly like XML, but it can be used to represent arbitrary data structures. Do you have any objections now?
      - benibela 7y ago
        That is only valid if the XDF standard has been published by the W3C
        gnulinux 7y ago
        This doesn't make any sense. W3C doesn't restrict/define use cases of XML, it defines structure and semantics thereof; using it as-if it is an XDF document is perfectly ok, just like you can use XML data for your DASH stream etc... It's a structure on top of XML structure.

theamk 7y ago

One thing this misses in the "dictionary" example is that tools (like xpath) push you towards "key in attribute" selection. One of the most common operations we do with dictionaries is lookup by known key, and storing the key in attributes makes it much easier.

spiralx 7y ago
"Key in attribute" is the correct way to do it, it's just that his examples are absolutely terrible and make no sense at all. A completely unstructured list of key-value pairs is overkill for any structured data format.

deleted 7y ago

[deleted]

zippergz 7y ago

If a piece of technology is "almost always" misused, is that the fault of the users, or the technology?

dimitar 7y ago
Exactly, this is some variant of No true Scotsman
deleted 7y ago
[deleted]
scarygliders 7y ago
If you mean "users" as in "users of the XML format", then it's the fault of developers as 'users' of their applications don't have any choice in the matter.
- zippergz 7y ago
  I mean "users" exactly the same way "used" was meant in the author's statement. So yes, the developers, who are the people who are using XML.
Gibbon1 7y ago
Marketing usually.

deleted 7y ago

[deleted]

deleted 7y ago

[deleted]

alkonaut 7y ago

I’m going to just argue the exact opposite of the article: xml and json are both structured data formats useful for tree like data graphs, such as objects. Whether that was the intended purpose when xml was designed is irrelevant. It’s what xml is used for in almost every case. The author also doesn’t suggest what should be used instead to encode structured data, or perhaps more importantly what should have been used to encode graph like things such as map/lists/objects in the 2000’s. Json really hasn’t been an alternative until quite recently (10 years ago?). In fact reading the article carefully I fail to see the author argue why xml shouldn’t be used as a data format either.

miggol 7y ago
I'd argue that for data storage purposes, you'd like to have a low metadata to information ratio. In the examples the author gives this seems to be the main problem, with way more characters being used for markup than for content. Compare that to JSON or TOML, which are more human-friendly and waste fewer bytes on structure to convey the same information. When used for data storage, two XML files of the same schema describing two completely different objects are likely to share a large amount of content, which is wasteful and gets in the way.
- RantyDave 7y ago
  I beg to differ. For a start, XML compresses well and besides, storage is monster cheap these days. XML is a better storage format because it documents what the data is (a title, a reference etc) as well as the data itself. There are many better reasons to hate XML.
  - hnick 7y ago
    XML does compress well as text or over the wire but the parsing trees can be quite large in memory and processing consumption. At least in Perl I've had enough scripts crash out due to this overhead when implementing the common/naive solution using off the shelf modules. You can get around this by choosing between DOM or SAX but I consider that a symptom of the problem, you choose XML to solve a problem and now you have another problem to solve.

altmind 7y ago

Can I have my gripe with apple plists? <key>CFBundleDisplayName</key> <string>TextEdit</string> <key>NSHumanReadableCopyright</key> <string>Copyright 2019</string> This may be perfectly parsable by a SAX parser storing some state, but its totally not processable by xslt.

k__ 7y ago
How old is this format? I had the impression mobile was some new try at frontend tech, but somehow iOS and Android threw a whole bunch of outdated stuff at me. I mean, before the iPhone I designed a XML based ETL config system and tried to avoid all the common XML errors, then I start doing a mobile app 10 years later and it's like all that knowledge was forgotten...
mikl 7y ago
Yeah, that's awful XML. Ironically, they used to have a decent readable format in NeXTStep, it was only later it was XML-ified.
nwellnhof 7y ago
> but its totally not processable by xslt. It's a bit inconvenient but perfectly processable with XPath: /dict/key[.='CFBundleDisplayName']/following-sibling::string[1]
spiralx 7y ago
I remember when Neverwinter Nights 2 came out touting that all of its data files were in XML for ease of modifying in user extensions. So I had a look and was it XML? Was it fuck, it was like a GeoCities novice's idea of how to code HTML - absolutely unparseable in any way unless you're the idiot who decided what the codebase needed was another shitty homegrown parser. And yes, plists suck and make your XPath selectors ugly, although you could write a function to abstract them out.

mojuba 7y ago

That's a very useful and sobering view actually. Unfortunately for the XML format it wasn't designed to prevent its own abuse. But anyway, XML is for documents sounds like a good and acceptable paradigm. That being said, one subtle and important (and often overlooked) difference between XML and, say JSON is that you can stream XML while parsing it on the application level, whereas JSON can not be parsed by the application due to arbitrary ordering of keys. (Of course lower level parsers use streaming anyway, but that's not the point) In fact you not only can but you should parse XML while streaming it. This is another common abuse: wherever you look you see some high level function that loads an entire parsed XML structure into memory at once. But once you start asking yourself where the file may be coming from you realize that your system may be open to denial of service attacks. E.g. is your system ready to receive a 16GB XML file?

Bouncingsoul1 7y ago
Exactly this, last time I checked the dump of german wikipedia was around 16GB so you'll find these files in the wild
fanf2 7y ago
There is no reason you can’t do a SAX-style parser for JSON. A quick bit of searchengineering will find dozens, eg. https://github.com/dgraham/json-stream https://github.com/dgraham/json-stream
- mojuba 7y ago
  You can in principle, but because the order is not guaranteed, you may find yourself accumulating things in memory. It depends on the task of course, but neither standard parsers nor generators are required to send adjacent JSON entities in a specific order. I remember someone describing a trick where instead of sending a JSON array they'd just send a stream of JOSN objects, one per line, so that the receiving end could parse the data in a streaming fashion. But that's not JSON anymore.
  - hnick 7y ago

miggol 7y ago

The worst example of this that I deal with regularly has to be the libvirt domain XML. https://libvirt.org/formatdomain.html https://libvirt.org/formatdomain.html It does occasionally put information outside of the tags, but because there's no logic to when, it's nearly worse.

tehjoker 7y ago

It is difficult for me to see what the real issue is with examples given. It seems to be more an aesthetic preference of the author rather than a technical argument. People can use formats for whatever they want. :P If you told me that the transmission and parsing rate is too slow for their application, that's a real dig at it.

mehrdadn 7y ago
I think it's a bit like trying to explain why a Python REPL isn't a substitute for a calculator? Like it can of course do what you want, and you can't "see" what's wrong if you just take what you see literally (you'll obviously get the same answers regardless of what tool you use), but it's just... not meant for that.

kbenson 7y ago

So, what is XML good for? If it's not good for data as everyone says (and I'm not inclined to argue), but it is good for documents, what kind of documents are we referring to? A defined metadata on a text document? A template used with data to generate something else? Is a configuration file a document or data? Where would I want to use XML that something like JSON, a text document, or some combination thereof wouldn't be better? I'm not being facetious, this is an honest question. Where are the "right" places to use XML?

jerf 7y ago
I wrote this four years ago: https://news.ycombinator.com/item?id=11446984 https://news.ycombinator.com/item?id=11446984 There are points of disagreement between me and the author, although I wouldn't get too passionate about them. Super-short version, reading over it again, is that XML is very good at what it does, but it really ought to be seen as a relatively specialized data format. It's really good at certain tasks, best-of-breed for a couple of them, and degrades rapidly as you get away from that. JSON is a fairly cheap & fast general-purpose format that's OK at a lot of things, isn't necessarily great at much, but as you get into more specialized use cases, also tends to degrade. Being a general-purpose format, perhaps arguably it degrades more "slowly", but it does degrade. Properly understood, IMHO, their use cases don't overlap much if at all, and the combination of them may cover a lot of space, but are still far, far from the only serialization formats you'll ever need.
TallGuyShort 7y ago
Think of semi-structured documents, where you have a list of pre-defined sections. I've seen it in use by insurance companies for case reports, in real estate for appraisals. And of course we've all seen it work well in the form of HTML. There's some structure to all of these examples, but mostly for annotating text sections. You need some flexibility built into the schema to add fields as needed, but you're not dealing with various map / list / primitive data types as a matter of routine. Just making this one up, but if LaTeX wasn't already the standard, I'd also use it if I was digitizing the content academic papers, for instance. You have an a header with metadata, abstract, the body, citations. There's some structure, a need to add some metadata, perhaps flexibly over time, but mostly it's just a document.

benibela 7y ago

The worst XML use I have ever seen are lists generated by Lazarus. Every list. For example in the project files you have: <RequiredPackages Count="5"> <Item1> <PackageName Value="LazUtils"/> </Item1> <Item2> <PackageName Value="treelistviewpackage"/> </Item2> <Item3> <PackageName Value="internettools"/> </Item3> <Item4> <PackageName Value="LCLBase"/> <MinVersion Major="1" Release="1" Valid="True"/> </Item4> <Item5> <PackageName Value="LCL"/> </Item5> </RequiredPackages>

sedachv 7y ago
Compare that to SGML: https://leancrew.com/all-this/2014/09/sgml-nostalgia/ https://leancrew.com/all-this/2014/09/sgml-nostalgia/ SGML really needs a revival.
hyperman1 7y ago
I once got a really nice one: Software v1.0: Config is a binary blob and everybody curses the nasty editor provided by vendors. Software v2.0: Config is in XML. Thank god! Oh wait. <Binaryblob><Byte value="65"/><Byte value="99"/> ....

robofanatic 7y ago

I guess the correct answer depends upon the requirement. I like this way <root> <item key="name">John</item> <item key="city">London</item> </root> So I can use this xpath to get the person's name: //root/item[@key="name"]/text() Not sure what would be the xpath to get the name if the XML was <root> <item> <key>Name</key> <value>John</value> </item> <item> <key>City</key> <value>London</value> </item> </root> This is a better example: <employees> <employee id="1"> <field name="name">John</field> <field name="city">London</field> </employee> <employee id="2"> <field name="name">Jack</field> <field name="city">Boston</field> </employee> <employees>

scandox 7y ago
If you accept the author's premise then all of these (including his) are wrong uses of XML.
elFarto 7y ago
The XPath for the second one would be: //root/item/key[text()="Name"]/../value/text()
- icebraining 7y ago
  Alternatively //root/item[key[text()="Name"]]/value/text()
  - benibela 7y ago
    You usually do not need to use text() in xpath, so this should work the same: //root/item[key="name"]/value
commandlinefan 7y ago
Well, actually, an even better example is: <employees> <employee id="1"> <name>John</name> <city>London</city> </employee> <employee id="1"> <name>Jack</name> <city>Boston</city> </employee> I know what you’re trying to do there - you’re trying to “future-proof” your schema by allowing introduction of arbitrary new elements. Which means that there’s no standard way to guard against somebody omitting a required field (like “name”) or adding a new field like “creditCardNumber” - other than to document your acceptable key values in a non-standard format and add defensive code that a validating parser would have given you. You’re better off taking as much advantage of the format as you can.

bullen 7y ago

I would love to know what people think of my XML node-graph/tree editor I made before JSON became mainstream (my excuse): http://rupy.se/logic.jar http://rupy.se/logic.jar It basically names the tag what you name the node. :S - You link/unlink nodes (I called them entities! Xo) by right-click-dragging between them. - You copy stuff by right-click-dragging to an empty space. - You delete by grabbing something by left-click-holding and pressing the delete key. - Oh, and nodes are completely tree structure expandable, just drag-drop attributes on nodes and nodes inside nodes. The editor uses lightweight rendering so you can have a ton of elements with good performance. (I know, not super intuitive; but very handy once you know about these.)

davidw 7y ago

I love the quote I originally saw on the Nokogiri (Ruby XML lib) site: "XML is like violence – if it doesn’t solve your problems, you are not using enough of it."

scarygliders 7y ago

I have always thought that using XML as a format for storing and retrieving configuration files, was complete insanity. Which is why I still use the simple, effective, INI format for configuration files for applications I write. XML for config files is madness personified.

deanCommie 7y ago

I think that Software Engineers should take influence from Authors (after all, are we not all craftsmen/artisans?) and incorporate the philosophies from https://en.wikipedia.org/wiki/The_Death_of_the_Author https://en.wikipedia.org/wiki/The_Death_of_the_Author The idea, for those not familiar, is that once a work of art is published (a novel, a poem, a song, a painting), it speaks for itself, and authorial intent no longer matters. That is, meaning and purpose are in the eye of the beholder/consumer. And there is no right or wrong way to "interpret" art. If someone finds meaning that the author did not intend, it is just as valid as a deeply hidden but intentional allegory they intentionally placed in when they were writing. The relevance to software is it applies to APIs, specifications, standards and formats. There is no such thing as users using your software or specification "wrong" - if they insist on doing so, the meaning has evolved. Evolve with it or die.

rkagerer 7y ago
That's a little extreme but you raise a good point. I think a talented spec designer anticipates how their work might be interpreted / used / abused, and, like an adroit villain, nudges their audience toward tenets their grand scheme seeks to achieve.

hashberry 7y ago

> a simple test for determining if an XML schema is well designed: remove all tags and attributes from it ... If what you have left over does not make sense ... you shouldn't be using XML at all. Magento 2 (acquired by Adobe for $1.68bn) uses XML to render its layouts. Here's some fun XML for the checkout page: https://github.com/magento/magento2/blob/2.3-develop/app/code/Magento/Checkout/view/frontend/layout/checkout_index_index.xml https://github.com/magento/magento2/blob/2.3-develop/app/cod...

unilynx 7y ago

XML is a actually wonderful format for data and especially extensible configuration if you combine it with XML Schema and CSS selectors... - XML schemas give you a ready-to-use format to describe, restrict and document available configuration settings. The unique keys help and a libxml2 gives ready to use validation, even if you may need to 'translate' its error messages before showing them to end users - XML schemas also support other annotations so you can further generalise your configuration readers by recording the necessary bindings in the XML schema itself, allowing to use it eg to define application user interfaces. - Almost any text editor can do basic syntax validation preventing most typographic errors, and even better if they can read the schema - XML schemas are extensible using <import>s, but namespaces still enforce some separation. You can define explicit points where plugins extend your configuration format using <any> - Human editable - closing tags are noisy but more readable than }],{}] when non-programmers may have to edit these files just to add a few extra textfields to an UI. - Better datatype support, eg datetimes, by using XML schemas. JSON's type support is too limited - Support for comments! - And once you've verified the schema... CSS selectors and DOM APIs to actually process the XML documents. YAML fixed quite a few things, but still no date times or as far as I know standardised approaches to defining schemas. And I've lost count at how any attempts exist to add schema information or namespacing to JSON... But for markup... we may be better off to just use markdown inside CDATA blocks

sedachv 7y ago
> XML schemas give you a ready-to-use format to describe, restrict and document available configuration settings. As someone that likes and uses s-expressions, I never thought I would find myself defending XML, but here we are in 2019, no one understands basic parsing theory anymore, and file formats have "evolved" to hot garbage like YAML and TOML. XML has some great tools in comparison: http://xmlsoft.org/xmllint.html http://xmlsoft.org/xmllint.html https://relaxng.org/ https://relaxng.org/ > But for markup... we may be better off to just use markdown inside CDATA blocks SGML can still make a comeback: https://leancrew.com/all-this/2014/09/sgml-nostalgia/ https://leancrew.com/all-this/2014/09/sgml-nostalgia/
- novok 7y ago

mickduprez 7y ago

XML is/can be much more than a markup language and yes, it can be used very badly but this is usually by inexperienced 'data wranglers' who don't understand the difference between data attributes and data proper. While XML can seem cumbersome (compared to JSON say) it is a very good 'data transport' tool when used correctly with a sensible schema (XSD). For example, we use XML as a 'vendor neutral' data format to export/import CAD geometry and associated data for town utilities such as buildings, pipes, roads etc. All this data has to be validated against the schema to ensure its correctness. Using a schema like this enables the city council to import this XML into the GIS system to be used for asset management, financial planning etc. A good schema can be key to sharing XML effectively between departments/applications and being a markup language this data can also be viewed independently using XLST.

h2odragon 7y ago

I present, in the spirit of 'worst XML ever', the docs for ScriptXML: https://www.egosoft.com:8444/confluence/display/XRWIKI/Mission+Director+Guide https://www.egosoft.com:8444/confluence/display/XRWIKI/Missi... This is the language used in the game X4 Foundations (and others in the series). An example of its use (mine, i claim no grace in it): https://github.com/h2odragon/dragoncommands/blob/master/aiscripts/deployglobe.xml https://github.com/h2odragon/dragoncommands/blob/master/aisc... ... XML is not a great format for an extension language, I have to say.

Gibbon1 7y ago
Eclipse project files are also hot garbage.

LameRubberDucky 7y ago

For those wondering what you do with XML as a document markup language, see the XML document that is the specification for XML. I had to look at the page source to determine it really is an XML document. Looks like an HTML document. https://www.w3.org/TR/xml/REC-xml-20081126.xml https://www.w3.org/TR/xml/REC-xml-20081126.xml

yellowapple 7y ago
Jesus. That's slick. Almost makes me appreciate all the bells and whistles of XML. Almost.
spiralx 7y ago
I've got a stylesheet I wrote that turns HTML or XHTML into a fully indented and highlighted representation of itself which I was quite proud of :) I used to spend a lot of time writing XSLT and XQuery lol.

commandlinefan 7y ago

The first rule of XML is: whatever you're doing with it, that's not what it was for.

micimize 7y ago

I can't find much to corroborate this article's take. RDF is a stark counter-example - a standard from the W3C. It has endorsement from Tim Bray, one of XML's co-authors.

tannhaeuser 7y ago

> In 1996, XML was invented. XML wasn't an original invention; it is specified as a proper SGML subset. From the XML spec: > The Extensible Markup Language (XML) is a subset of SGML that is completely described in this document. Now I totally agree that SGML and XML aren't for service payloads and config files. The sole purpose of markup languages is representing structured text. And arguably, SGML fills this role much more adequately than XML today as it can represent (via the SHORTREF mechanism) custom Wiki syntaxes such as markdown and others, and in contrast to XML, can deal with the largest corpus of markup out there eg. can parse HTML with all its minimization features such a omitted tags, enumerated and unquoted attributes, etc. See [1] for a practical introduction (disclaimer: link to a tutorial I held last month at ACM DocEng). [1]: http://sgmljs.net/docs/sgml-html-tutorial.html http://sgmljs.net/docs/sgml-html-tutorial.html

catalogia 7y ago
SGML's implied close tags are a pain in the ass though. End tags in XML are overly verbose, but at least they're required.
- tannhaeuser 7y ago
  You control whether an element requires start- and end-element tags in your element declaration via "O" (letter O as in "omissible") in the respective tag omission indicator position: <!ELEMENT e - -(f,g,h) -- no tag omission --> <!ELEMENT f O - (#PCDATA) -- start-tag omission --> <!ELEMENT g - O (#PCDATA) -- end-tag omission --> <!ELEMENT h O O (#PCDATA) -- both start- and end- tag omission allowed--> What's painful about end-element tag omission?
- gnuarch 7y ago
  Can't find the source, but vaguely remember this line from a Usenet or mailing list post, maybe by Tim Bray: The lack of </> in XML is a crime against humanity.

just_myles 7y ago

I agree with this portion 100% The correct way to express a dictionary in XML is something like this: <root> <item> <key>Name</key> <value>John</value> </item> <item> <key>City</key> <value>London</value> </item> </root> In the past I used to create scripts that exported xml from relational data but didn't really understand the right way to build and structure them.

billsix 7y ago

RIP Eric Naggum http://www.schnada.de/grapt/eriknaggum-xmlrant.html http://www.schnada.de/grapt/eriknaggum-xmlrant.html

Quarrelsome 7y ago

I once had to write a data layer in xml, in-situ with a lifespan for up to hours, as more data was appended to it. An invalid xml document that you couldn't load in many xml apis for 99.9% of its lifespan. I begged and pleaded the lead architect to use an sqlite db for the elements of the data until the transaction was complete and then merely produce the xml file at the end, but no. It had to survive power outs too.

lmilcin 7y ago

And JavaScript was never meant to be used to build applications... There are many, many more inventions that are used for different purposes they were meant for. The Internet was created so that US can withstand nuclear attack and it was never meant to be primarily used to spread advertisements. Get over it.

billpg 7y ago

XML with only tags and attributes (nothing but ignored spaces between '>' and '<') is a reasonable structured data format.