8 ms·

XML is notoriously expensive to properly parse in many languages. Basically, the entire world centers around 3 open source implementations (libxml2, expat and

• necovek 8mo ago
  FWIW, this is also one of the reasons MathML has never become the "input" language for mathematics, and the layout-focused (La)TeX remains the de-facto standard. Ergonomics of input are important because they increase chances of it being correct, and you can usually still keep it strict and semantic enough (eg. LaTeX is less layout-focused than Plain TeX)
  • phlakaton 8mo ago
    But there, as with any DSL, you are trading-off ease of expression with ease of processing (e.g. interiperability). Every embedded DSL, XML included, chooses some amount of ease of processing.
  • rhdunn 8mo ago
    MathML is used a lot in standards/publishing, such as with JATS and EPUB. MathML is also natively supported in the HTML specification.
• raverbashing 8mo ago
  > and have two axes for adding metadata: one being the tag name, another being attributes Yes let's not even get started on implementations who do <something value="value"></something>
• sriku 8mo ago
  While this can give a notation for the domain, you'd still need an engine to process it. Prolong+CLPFD perhaps meets it well (not too familiar with the tax domain) and one could perhaps paraphrase Greenspun's tenth rule to this combo too.
• petcat 8mo ago
  > XML Is a Cheap [...] > XML is notoriously expensive to properly parse in many languages. I'm glad this is the top comment. I have extensive experience in enterprise-y Java and XML and XML is anything but cheap. In fact, doing anything non-trivial with XML was regularly a memory and CPU bottleneck.
  • bubbleRefuge 8mo ago
    Yup. SAP and their glorious idocs with german acronyms
  • diffuse_l 8mo ago
    That's if you parse the into a DOM and work on that. If you use SAX parsing, it makes it much better regarding the memory footprint. But of course, working with SAX parsing is yet another, very different, bag of snakes. I still hope that json parsing had the same support for stream processing as XML (I know that there are existing solutions for that, but it's much less common than in the XML world)
  • alexpetros 8mo ago
    In the context of the article, "cheap" means "easy to set up" not "computationally efficient." The article is making the argument that there are situations in which you benefit from sacrificing the latter in favor of the former. You're right that it's annoyingly slow to parse though and that does cause issues I'd like to fix.
  • hrmtst93837 8mo ago
    [flagged]
    • sparkie 8mo ago
      (Properly formatted) XML can be parsed, and streamed, by a visibly-pushdown automaton[1][2]. "Visibly Pushdown Expressions"[3] can simplify parsing with a terse syntax styled after regular expressions, and there's an extension to SQL which can query XML documents using VPAs[4]. JSON can also be parsed and validated with visibly pushdown automata. There's an interesting project[5] which aims to automatically produce a VPA from a JSON-schema to validate documents. In theory these should be able outperform parsers based on deterministic pushdown automata (ie, (LA)LR parsers), but they're less widely used and understood, as they're much newer than the conventional parsing techniques and absent from the popular literature (Dragon Book, EAC etc). [1]:https://madhu.cs.illinois.edu/www07.pdf https://madhu.cs.illinois.edu/www07.pdf [2]:https://www.cis.upenn.edu/~alur/Cav14.pdf https://www.cis.upenn.edu/~alur/Cav14.pdf [4]:https://web.cs.ucla.edu/~zaniolo/papers/002_R13.pdf https://web.cs.ucla.edu/~zaniolo/papers/002_R13.pdf [3]:https://homes.cs.aau.dk/~srba/courses/MCS-07/vpe.pdf https://homes.cs.aau.dk/~srba/courses/MCS-07/vpe.pdf [5]:https://www.gaetanstaquet.com/ValidatingJSONDocumentsWithLearnedVPA/ https://www.gaetanstaquet.com/ValidatingJSONDocumentsWithLea...
• alexpetros 8mo ago
  Author here. I agree with all this, and I think it's important to note that nothing precludes you from doing a declarative specification that looks like imperative math notation, but it's also somewhat besides the point. Yes, you could make your own custom language, but then you have created the problem that the article is about: You need to port your parser to every single different place you want to use it. That's to say nothing of all the syntax decisions you have to make now. If you want to do infix math notation, you're going to be making a lot of choices about operator precedence. The article is using a lot of simple functions to explain the domain, but we also have switch statements—how are those going to expressed? Ditto functions that don't have a common math notation, like stepwise multiply. All of these can be solved, but they also make your parser much more complicated and create a situation where you are likely to only have one implementation of it. If you try to solve that by standardizing on prefix notations and parenthesis, well, now you have s-expressions (an option also discussed in the post). That's what "cheap" means in this context: There's a library in every environment that can immediately parse it and mature tooling to query the document. Adding new ideas to your XML DSL does not at all increase the complexity of your parsing. That's really helpful on a small team! I agonized over the word "cheap" in the title and considered using something more obviously positive like "cost-effective" but I still think "cheap" is the right one. You're making a cost-cutting choice with the syntax, and that has expressiveness tradeoffs like OP notes, but it's a decision that is absolutely correct in many domains, especially one where you want people to be able to widely (and cheaply) build on the thing you're specifying.
  • johnbarron 8mo ago
    Why did you hardly engaged in the article on the subject of schema driven validation?
    • alexpetros 8mo ago
      This is a good question! We do it, it works, and it's definitely an advantage of XML over alternatives. I just personally haven't had the time to dig in and learn it well enough to write a blog post about it. In practice I think people update the Fact Dictionary largely based on pattern matching, so that's what I focused on here.
• Someone1234 8mo ago
  I keep seeing people make the same mistake as XML made over and over; without learning from it. I will clarify the problem thusly: > The more capabilities you add to a interchange format, the harder that format is to parse. There is a reason why JSON is so popular, it supports so little, that it is legitimately easy to import. Whereas XML supports attributes, namespaces, CDATA, DTDs, QNames, xml:base, xml:lang, XInclude, etc etc. They gave it everything, including the kitchen sink. There was a thread here the other day about using Sqlite as an interchange format to REDUCE complexity. Look, I love Sqlite, as an application specific data-store. But much like XML it has a ton of capabilities, which is good for a data-store, but awful for an interchange format with multiple producers/consumers with their own ideas. CSV may be under-specified, but it remains popular largely due to its simplicity to produce/consume. Unfortunately, we're seeing people slowly ruin JSON by adding e.g. commands to the format, with others than using those "comments" to hold data (e.g. type information), which must be parsed. Which is a bad version of an XML Attribute.
  • xienze 8mo ago
    > Whereas XML supports attributes, namespaces, CDATA, DTDs, QNames, xml:base, xml:lang, XInclude, etc etc. They gave it everything, including the kitchen sink. But you don't have to use all those things. Configure your parser without namespace support, DTD support, etc. I'd much rather have a tool with tons of capabilities that can be selectively disabled rather than a "simple" one that requires _me_ to bolt on said extra capabilities.
    • cbm-vic-20 8mo ago
      As a data interchange format, you can only depend on the lowest commonly implemented features, which for XML is the base XML spec. For example, Namespaces is a "recommendation", and a conformant XML parser doesn't need to support it.
      • smashed 8mo ago
        The problem comes when malicious actors start crafting documents with extra features that should not be parsed, but many software will wrongly parse them because they use the default, full featured parser. Or various combinations of this. It's a pretty well understood problem and best practices exist, not everyone implements them.
• twoodfin 8mo ago
  Much of XML’s complexity derives from either the desire to be round-trip compatible with any number of existing character and data encodings or the desire to be largely forward-compatible with SGML. A parser that only had to support a specified “profile” of XML (say, UTF-8 only, no user-defined entities or DTD support generally) could be much simpler and more efficient while still capturing 99% of the value of the language expressed by this post.
  • phlakaton 8mo ago
    That's besides the point of this post. You're welcome to enforce such a profile on your documents, but the point of this post is the ease from throwing the whole ecosystem of out-of-the-box XML tools at it, tools which don't assume any such profile. (Now ITOT they may have implicit or explicit profiles of their own, e.g. where safe parsing, validation, and XSLT support are concerned, but they have a large overlap.)
    • twoodfin 8mo ago
      Indeed, I was agreeing that the XML ecosystem as currently constituted has all the problems necovek pointed out. But the W3C might have made some different choices in what to prioritize—notably, identifying a common “XML: The Good Parts” profile and providing the standards infrastructure for tools to support such a thing independent of more esoteric alternatives for more specialized use cases like round-tripping data from French mainframes. Instead they chased a variety of coherent but insufficiently practical ideas (the Semantic Web), alongside design-by-committee monsters like XHTML, XSLT (I love this one, but it’s true), and beyond.
• phlakaton 8mo ago
  Unless you are compiling really large systems of DSL specification, speed of parsing is not the operation you want to be optimizing. XML for this use case, even if you DOM it, is plenty fast. What are more concerning are the issues that result in unbounded parses – but there are several ways to control for this.
  • Hendrikto 8mo ago
    > XML for this use case, even if you DOM it, is plenty fast. This mindset is why we have computers now that are three+ orders of magnitude faster than a C64 but yet have worse latency.
    • phlakaton 8mo ago
      Interesting you should complain about that with a legacy technology that's almost 30 years old (or 50 years old if you count SGML). In particular, XML has gotten no more complex or slow than it was 20 years ago, when development largely stopped. For this application it's plenty fast. Even if you've got a Pentium machine.
• gchamonlive 8mo ago
  That's a strange comment... Cheap here is semantically different from cheap in the article. Here it means "how hard it hits the CPU" and in the article is "how hard it is to specify and widely support your DSL". You also posted a piece of code that the author himself acknowledged that is not bad and ommited the one pathological example where implementation details leak when translating to JavaScript. It just seems like you didn't approach reading the article willing to understand what the author was trying to say, as if you already decided the author is wrong before reading.
  • phlakaton 8mo ago
    Some people just comment on the title. Maybe that's what happened here.
  • necovek 8mo ago
    Nope, not cheap in my comment means expensive to implement: defining the XML schema, which has been done by someone else, and then using that schema properly, is what makes use of XML expensive (it is a lot of things to learn for more than one engineer in the team).
    • gchamonlive 8mo ago
      I misunderstood that part of the comment, sorry about that
• twic 8mo ago
  You don't even need to specify a DSL to make that code declarative. It can be real code that's manipulating expression objects instead of numbers (though not in JavaScript, where there's no operator overloading), with the graph of expression objects being the result.
• PantaloonFlames 8mo ago
  Your first counterpoint seems unnecessarily picky. > So while it is a suitable DSL for many things (it is also seeing new life in web components definition), we are mostly only talking about XML-lookalike language, and not XML proper. If you go XML proper, you need to throw "cheap" out the window. But the TWE did not embrace all that stuff. It’s not required for its purpose. And to call it “xml lookalike” on that basis seems odd. It’s objectively XML. It doesn’t use every xml feature, but it’s still XML. It’s as if you’re saying, a school bus isn’t a bus, it’s just a bus-lookalike. Buses can have cup holders and school buses lack cup holders. Therefore a school bus is not really a bus. I don’t see the validity or the relevance.
  • necovek 8mo ago
    As discussed in the thread, the author has not dove deep into schema validation, but the org does use it. Ignoring that part of schema definition and subsequent validation is exactly why it seems "cheap" on the surface. So, TWE is not using an XML lookalike language, but someone has done the expensive part before the author joined in.
• quotemstr 8mo ago
  > The main property of SGML-derived languages is that they make "list" a first class object, and nesting second class (by requiring "end" tags), As opposed to JSON, which famously lacks lists? What does "second class" even mean here? How is having an end-indicator somehow a demotion? > talking about XML-lookalike language, and not XML proper. If you go XML proper, you need to throw "cheap" out the window. libxml2 and expat are plenty fast. You can get ~120MB/s out of them and that's nowhere near the limit. Something like pugixml or VTD can do faster once you've detected you're not working with some kind of exotic document with DTD entities.
• tannhaeuser 8mo ago
  > The main property of SGML-derived languages is that they make "list" a first class object, and nesting second class (by requiring "end" tags) ... I think you're missing the forrest for the trees ;) The major point of SGML in this context is that elements have content models defined by regular expressions, just like any other grammar productions eg. BNF.
• electroly 8mo ago
  I shipped 20MB of XML with a product back in 2014; we loaded it at startup, validated it against the XSD, and the performance for this use case was fine. It was big because we did something kinda like what TFA suggests: I designed a declarative XML "DSL" and then wrote a bunch of "code" in it. We had lots of performance problems in that project, but the XML DSL wasn't the cause of any of them; that part was fine. I think "expensive" can mean a lot of different things. It was cheap in terms of development time and the loading/validation time, even on 20MB of XML, was not a problem. Visual Studio ships a tool that generates C# classes from the XSDs which was handy. I just wrote the XSDs and the framework provided the parsing, validation, node classes, and tree construction. This is as "XML proper" as I think it's possible to get. I don't believe that .NET's XML serializer uses any of the open source projects mentioned in your post, so maybe we just have especially good XML support in .NET. I think Java has its own XML serializer, too. I bet most XML generated and consumed in the world is not one of those three open source C/C++ libraries. I think Java alone might be responsible for more than half of it.