10 ms·

JSON vs. XML with Douglas Crockford

88 points by ZeljkoS 4y ago

moritzwarhier 4y ago
Again? I'll probably listen to this at some point and genuinely like hearing about the origins of JSON from Crockford himself. Apart from that, I'd be surprised if this adds anything to what I've already heard and read ad nauseam, including from Crockford himself. JSON is just a nice, simple serialization format that compresses well and interoperates acceptly with most programming languages. Most importantly with the language that it was designed to be consumed by: Javascript. If one needs better performance or better guarantees about serialized data types, I guess one should use Protobuf, and/or ASN.1 (was that even correctly spelled?) or any of the countless other formats I have no clue about. If you need better schema restrictions and interconnected entities + circular references: use a database, I guess. At least everything XML has done in this space has more or less failed (e.g. SOAP). Well, failed might be too strong a word. But all the work on defining data types and schema seems almost independent of the underlying syntax. Since I've now spent too much time on this uninformed first comment, I will now listen what Douglas Crockford wants to say. But I think I've already read this story in another form a couple of years ago. JSON is really a good example of "worse is better". E.g. an OpenAPI spec with automated client code and TypeScript definitions is not much different from SOAP on the surface level.
- wongarsu 4y ago
  Two things where imho XML performs well: 1: text with markup. Does not need all the XML machinery, but tags make a lot of sense here. 1b: Arbitrarily nested stuff 2: extensability by third parties without coordination. Namespaces as a central concept help out a lot here, and this use case can even justify some of the more insane features of XML (downloading remote schemas etc) If you don't need either of those things, you won't get much value out of XML. If you do need them, XML can be neat. Certainly easier than trying to allow people to add arbitrary data (with validation) to your protobuf. It's just that these use cases are somewhat rare
  - halayli 4y ago
    More importantly, streaming.
  - XorNot 4y ago
    I was recently downloading and parsing the US Patent application database, which is emitted in XML. In 2023, Python lxml can't handle multiple documents per file. And no one has any suggestions on how to handle this other then "break on the new document tag". Other libraries didn't look promising in other languages. Then if you try to read it, you need to disable all the DTD stuff because it immediately tries to open other local files on disk...which it can't find. There's some corruption in how it parses DTDs because I had to implement a custom DTD handler in my jupyter-notebook to read it. I could blame the dominant XML processing library of Python, I could blame the lack of attention XML gets, but really? This is just another contribution in a long line of XML not working in some way in every language as soon as you're not using that languages specific XML flavor.
    - dehrmann 4y ago
      I'm curious how Java would handle this. Its XML support is usually very mature. Java and JSON is a different story. There was a window when the only JSON parser Java shipped with was JSON.parse() in its Javascript interpreter.
      - DonHopkins 4y ago
        "Java is a DSL for taking large XML files and converting them to stack traces." -Andrew Back https://www.reddit.com/r/programming/comments/eaqgk/java_is_a_dsl_for_taking_large_xml_files_and/ https://www.reddit.com/r/programming/comments/eaqgk/java_is_... "I Wanna Be <![CDATA[" Sung to the tune of “I Wanna Be Sedated”, with apologies to The Ramones. https://donhopkins.medium.com/i-wanna-be-cdata-3406e14d4f21 https://donhopkins.medium.com/i-wanna-be-cdata-3406e14d4f21
    - nrclark 4y ago
      I haven't had to work with XML all that often. For the few programs I've made that need to speak XML, I've always used the XML module in Python's stdlib. Is that bad practice? What does lxml provide as a benefit?
- 2h 4y ago
  > If one needs better performance or better guarantees about serialized data types, I guess one should use Protobuf ProtoBuf is a horrible format. its not self describing like JSON, when it easily could be, without costing any extra bytes.
  - emptysea 4y ago
    aren't there some ambiguities that would require extra data to decipher? like if I have a message inside of a message vs just a bytes field that happens to have a valid protobuf?
    - 2h 4y ago
      Exactly, they could have added an extra wire type to signify messages without costing any extra bytes. Nothing is stopping ProtoBuf from being self describing except incompetence.
- paulddraper 4y ago
  > At least everything XML has done in this space has more or less failed (e.g. SOAP) Bwhahahaha good one SVG DOCX SAML JSX The legacy of XML incalculable and expanding.
  - jeremyjh 4y ago
    XLSX. I'd guess around 80% of the world's code is inside XML.
  - throwawaymaths 4y ago
    Svg, docx, and jsx are not in the data interchange business. All three are actually human-consumed documents.
    - recursive 4y ago
      Who's talking about interchange? What data interchanged is not human consumed? Where did those goalposts go?
      - throwawaymaths 4y ago
        JSON is human readable but mostly not human consumed. Something like parquet or protobuf is neither human readable nor human consumed. Xml excels when the content must be human consumed and machine readable.
  - deleted 4y ago
    [deleted]
  - moritzwarhier 4y ago
    Yeah I had SVG in mind when I wrote that post but I thought of different use cases. Of course every document format can also be called a serialization format. I enjoy JSX as well (and even SOAP!). That's why I weakened the quoted sentence in my comment. When I hear "JSON vs XML" I can't help but mainly think of data payloads in web applications, especially non-static client/server communication. I wouldn't want JSON instead of SVG either :)
- DonHopkins 4y ago
  It's worth listening to, or at least reading the transcript, just for the Dave Winer burn. (And so much more!) >Adam: That was it. That was the creation of JSON, which everyone is using today. But back then, everyone rejected it. >In some ways it was a marketing problem. On one side, you had Doug, trying to convince customers that they can build interactive applications on the web using JavaScript and this simple thing called JSON. But on the other side, you had XML that had these big companies behind it, IBM, Microsoft, and big consultants. And later they even had some tech influencers like Dave Winer. >Douglas: He’s someone who should have known better. He had a website called scripting.com. His style of scripting came from a clever program that he had written for the Macintosh called Frontier, in which he had a scripting language and an outliner and a word processor and a database, all in one program. And the idea was that you could do virtually anything in Frontier with a little bit of scripting. And he was also one of the big promoters of SOAP, the Simple Object Annoying Protocol. >I don’t remember what the A was, but it might have been atrocious or abominable, I don’t know. >But SOAP was a big deal at the time. They were right in wanting simplicity. They didn’t accomplish it, but they put simplicity in the name as sort of an aspirational thing. And so, when I started showing how JSON works, he was really threatened by that. And on his website, which was well-read at the time, he complained that, “this isn’t even XML. We should find who did this in string them up now”, which was a really ugly thing to say. >Fortunately, nobody listens to Dave Winer, so I’m still here. Dave's done some brilliant influential stuff, which Doug credits and I've written about before, but being annoying is Dave's brand, so Doug's "SOAP" joke is dead on. It's just as fair as referring to Marc Canter's "People Aggregator" as "People Aggravator". https://news.ycombinator.com/item?id=20780928 https://news.ycombinator.com/item?id=20780928 https://news.ycombinator.com/item?id=21170440 https://news.ycombinator.com/item?id=21170440 https://news.ycombinator.com/item?id=16224154 https://news.ycombinator.com/item?id=16224154 https://www.gawker.com/183778/people-aggravator-marc-canter-gets-lonely-spams-for-friends https://www.gawker.com/183778/people-aggravator-marc-canter-...
  - moritzwarhier 4y ago
    I did listen to the podcast lateron and was pleasantly surprised by the breadth of topics, although I have to admit I fell asleep halfway through (not because of the contents). Thanks a lot for the money quotes! Casual conversation in English is still sometimes hard to listen to for me without focusing a lot.
deleted 4y ago
[deleted]
theschmed 4y ago
Or you could do what one API I just discovered does: embed escaped XML inside JSON! We truly live in a world
- breakfastduck 4y ago
  I have worked with a supplier / service who advertised us their brand new shiny REST/JSON based API that we could use to replace the drop a csv in a sftp workflow they offered previously. The API was a post with a single JSON field called data, with a CSV string value. You just know they were asked by management to deliver an API and decided to do quite literally the minimum viable. They had 'no plans' to expand it.
  - cgio 4y ago
    How anyone would see that as a MVP when you can make a pretty solid API with Postgrest eludes me.
    - crgwbr 4y ago
      It’s MVP because it (1) is an API and (2) is doing nothing more than writing it’s output to the same sftp server the client used to hit directly. Not debating it’s dumb, but almost by definition enterprise tech ossifies in weird, dumb ways.
  - hyperman1 4y ago
    I've seen similar in the wild with XML, except the charset was wrong and escaping was incomplete. So you get the XML header with UTF8 encoding, a root tag, an indented child tag. 3 lines. Then ye olde CSV, with the 5 required entities for ascii escaped, but if you manage to put accented chars in it, they end up in ISO 8859-1. Then 2 lines footer. If you wrote out the file, the header and footer had to be byte for byte identical. Entities encoding non ascii just crashed the parser. Of course, the CSV lines were sensitive to indentation, they just dropped the first 8 chars of each line. Still, XML was officially supported.
- icedchai 4y ago
chrisco255 4y ago
This interview is as much about the history of early web development and the serendipitous process by which JSON won out. After reading, it's apparent that JSON could have easily lost and been looked over entirely. Crockford's persistence in evangelizing it even after the company he worked at when it was created went bankrupt led to its eventual adoption.
- simplotek 4y ago
  > Crockford's persistence in evangelizing it even after the company he worked at when it was created went bankrupt led to its eventual adoption. I don't think this is true at all. The reason that JSON won is stated quite clearly in the interview: you do not need a software stack to handle JSON. This feature sells itself for any software development team who needs to have frontends running in a browser consuming data from a backend. You do not need an evangelist to convince people to use a feature that is ubiquitous, free, extensively tested, and more performance than anything in the market.
  - chrisco255 4y ago
    There's plenty of historical examples of sub-optimal standards developing enough traction and network effects that the whole industry is stuck with them for decades. Initially you did need to pull in a library to parse JSON, it wasn't part of the JS language until Crockford pushed for it (via his later position of authority at Yahoo working on YUI). Particularly in the early web, the need for async "liveness" of pages didn't exist, so pages didn't consume backends directly. Everything was handled on refresh or navigation and server rendered as HTML. It was only after AJAX-style web apps took off (post-Gmail, really), that JSON started to get real traction. There were no JSON parsers in C#, Java, Python, etc initially. XML was far more common and SOAP was generally used for cross-endpoint communication.
gnabgib 4y ago
Posted 16 days ago (152 points, 242 comments)[0], 17 days ago (4 points, 1 comment)[1], and twice 19 days ago (1 points)[2], (3 points, 1 comment)[3] [0]: https://news.ycombinator.com/item?id=35467711 https://news.ycombinator.com/item?id=35467711 [1]: https://news.ycombinator.com/item?id=35452861 https://news.ycombinator.com/item?id=35452861 [2]: https://news.ycombinator.com/item?id=35429976 https://news.ycombinator.com/item?id=35429976 [3]: https://news.ycombinator.com/item?id=35427394 https://news.ycombinator.com/item?id=35427394
- tough 4y ago
  Feels like someone wants XML back
  - teleforce 4y ago
    XML ship has sailed, hopefully CUELANG will be widely adopted for a better future.
    - fzeindl 4y ago
      Has it? Is there something akin to XSLT for JSON yet?
      - drzaiusx11 4y ago
        I've made do with .to_xml, use xlst on results, then .to_json. It definably works (with some caveats) but would love to cut out the middleman. I believe xlst 3.0 added explicit json support but I've yet to give it a try: https://www.saxonica.com/papers/xmlprague-2016mhk.pdf https://www.saxonica.com/papers/xmlprague-2016mhk.pdf
jgalt212 4y ago
It was a really good listen for the historical perspective alone. I did not know how much bad blood there was between Dave Winer and Crockford. Winer was a co-creator of XML-RPC, but REST and RPC are two different use cases. I guess they were not so clearly delineated at the time.
- denkmoon 4y ago
  In practice they're not so delineated today either. How often do you find RPCs labeled "REST-like API". Few "REST-like" APIs actually apply HATEOAS, etc.
  - kstrauser 4y ago
    “JSON over HTTP? Yep, we’ve got REST!”
  - osigurdson 4y ago
    I think true REST APIs should be called a HATEOS APIs these days to disambiguate. REST just means GET/POST json at this point.
  - Shorel 4y ago
    Aptly named HATE-something.
smitty1e 4y ago
Good listen at 1.5 speed.
smitty1e 4y ago
Summary: Crockford is a humble genius and worthy of emulation.
- naikrovek 4y ago
  humble? lol genius? lol he is certainly not humble and he's no smarter than anyone else. right place, right time. his LACK of humility is what gave him the persistence required.
  - smitty1e 4y ago
    While I have no inarguable proof to refute you, and maybe it's easy to sound cool when history has proven one correct, my characterization of "humble genius" is based on DC's low-key presentation within the podcast. At no point did I perceive him to assert any sort of personal superiority. While I use his JSON creation heavily, JavaScript as such is the second least attractive language, after PHP, in my view.
    - naikrovek 4y ago
      at no point did you think that him saying "I was right all along and everyone who opposed me was wrong" is not a message of personal superiority? giving that message is to claim superiority over others, whether it is intentional or not. this podcast is the most humble I have ever heard him, to be fair, but make no mistake; DC is not a humble person. A humble person would not have said "JSON was my/our idea" or "I was right"; a humble person would not be on a podcast talking about themselves at all.
jancsika 4y ago
Side note-- I can just feel the frontend dev's juice running out as they put the finishing touches on the volume slider on that audio player. It suspiciously only updates the attenuation on mouseup. Here's my hypothesis: they initially implemented it the obvious way on mousemove (or whatever similar immediate event). Then they a) noticed the horrible zipper noise that came from doing that, b) felt dread at the idea of plumbing into the webaudio API, c) said "aw fuck it" out loud and d) bound the event to mouseup. Does the dev of that framework post here? If so, am I close on this one? If so I can commiserate from the opposite end-- I spent a long time getting audio stuff like this just right and then fudging text alignment for lack of juice.
- tantalor 4y ago
  What causes the "horrible zipper noise"?
  - fathyb 4y ago
    Any changes in waveforms that are not smooth can cause percussive sounds (that's what percussions are). Playing audio suddenly causes a pop, and changing audio levels abruptly causes a weird zipper sound (the sound of a thousand MIDI drums playing in a few milliseconds). The reasons are very interesting: your brain does a Fourier Transform (sort of) in real time to listen to audio. It allows you to separate two people or two instruments producing sounds at the same time, and abrupt changes will cause this transform to product artefacts. The right way of doing it is to always make progressive changes to the gain level. This property is often used in audio engineering to "add or remove punch" in recordings: you can decrease the attack level to make the gain changes more aggressive and add punch, or lower it to make it smoother, over do it and it sounds muffled. In more concrete words: use the setTargetAtTime WebAudio API.
    - KMag 4y ago
      It's more the cochlea performing some kind of convolution of roughly sinusoidal wavelets with the audio signal, bringing the signal from temoporal to frequency domain, right? Does more neurological post-processing the cochlear output bring it closer to a Fourier transform?
thunderbong 4y ago
This paragraph caught my eye - > I think Object-Oriented Programming was an important advance over procedural programming, which is what we had been doing, but we’re stuck. We’re supposed to change paradigms every generation, and we haven’t. So we’re way behind now. Object-Oriented Programming hit the mainstream in 1980. We’re in the 2020s and we’re still there. We should have moved on.
- kagakuninja 4y ago
  Functional programming is the next big thing. Ideas from functional programming have gone mainstream, and are found in almost all major languages: lambdas, higher order functions, and immutability / purity. Monads have made some minor inroads (although they get bastardized by OO people, see Java and Javascript).
  - silverwind 4y ago
    Yes, but the remnants of OO still linger, both in people's minds and code.
    - klysm 4y ago
      Most importantly in people’s minds. A lot of early educational content is rooted in flawed software philosophy that is carried out into the world
    - usrusr 4y ago
      There are still remnants of ideological OOP use around and they are terrible indeed, but remnants on the language level are perfectly fine as long as they are not used as originally intended. Paraphrasing Crockford, we have found OOP, the good parts. It's used for structuring code now, not for modelling the world. That's a huge difference.
    - DonHopkins 4y ago
      Well if people wouldn't keep brutally deconstructing it [1] then serving it up as a delicious la carte menu [2] of tempting indulgences, maybe object programming will finally die! [1] https://www.info.ucl.ac.be/~pvr/decon.html https://www.info.ucl.ac.be/~pvr/decon.html [2] http://www.paulgraham.com/reesoo.html http://www.paulgraham.com/reesoo.html
paulddraper 4y ago
I find it hilarious whenever someone is treats XML as this archaic thing. When the hot new thing in web dev is... ... ... JavaScript XML (JSX)
- afavour 4y ago
  I was always under the impression JSX stood for JavaScript Syntax Extension. Whatever it is it certainly isn’t XML.
  - recursive 4y ago
    Whatever it is isn't too far from XML.
  - paulddraper 4y ago
    > JSX is an XML-like syntax extension to ECMAScript > https://facebook.github.io/jsx/ https://facebook.github.io/jsx/ So yes, it's not XML. It's JavaScript XML.
- Shorel 4y ago
  > Balanced named tags is a critical syntactic feature of the XML-style notation. This line made me think about the many discussions about Python syntax and the value of having brackets to know when a code block ends and starts. Balanced named tags are another step to make something easier to visually parse, at the cost of having much more text.
osigurdson 4y ago
I like json better but if I'm really honest about it, there isn't really that much difference.
- usrusr 4y ago
  Sounds like you have missed the "programming in XML" phase that triggered Crockford to do JSON: huge configuration languages, usually with almost good schemas. And nothing in there could ever fail before runtime. It was driven by a deep fear of programming, a desperate hope that maybe you could build something so configurable that you would reach a state of done after which you would never ever have to send the source through a compiler again. In reality however, often the best way to understand a problem in those configuration monsters (or the only way...) was to recompile the non-xml part with some tailored ad-hoc diagnostics. But the idea that the configuration was supposed to change a lot while the code is immutable proved surprisingly robust against ongoing demonstrations of exactly the opposite happening.
yawnxyz 4y ago
You can't spell Complexity without "xml"...
DonHopkins 4y ago
I love Douglas Crockford. >Douglas: Some of the people who were doing JavaScript, were doing it badly and didn’t want to be told that they were doing it badly. There is this notion among second rate programmers that the most important thing they do is express themselves. It’s not making programs that work well and are free of error. That’s way down on the list. What’s much more important is expressing themselves, that, I’m an artist and I express my arts by leaving out semicolons. It’s that kind of thing. So what I was saying did not resonate with that at all. So that audience was not happy to hear it, and JavaScript was just not being used.
DonHopkins 4y ago
>Yahoo Since when did Yahoo lose its "!"?
pkphilip 4y ago
I only wish that JSON had a slightly less verbose format for array types with records with fixed number of columns: example: instead of the following: [{"field1":"value", "field2":"value", "field3":"value"}, {"field1":"value1", "field2":"value2", "field3":"value3"}, {"field1":"value3", "field2":"value3", "field3":"value4"}] If you could have this instead: [{"columns": {"field1", "field2", "field3"}, "rows": [{"value1,"value2","value3"},{"value1,"value2","value3"},{"value1,"value2","value3"}] If json parser could then have the smarts to return the values by position for the various field names even though each row does not include the field names.
- RedShift1 4y ago
  If you use compression, it's a non-issue.
  - IggleSniggle 4y ago
    It’s still an issue for the “readable by humans” use case as it dramatically reduces readability?
- 5Qn8mNbc2FNCiVV 4y ago
  It's JavaScript Object Notation. An array of objects looks like that, if you want to have another data structure you can do it and have the accompanying code to consume it. The only thing holding you back from doing that is 1kb of JS as the second argument to JSON.parse
throwaway290 4y ago
> JSON is foundational stuff for the web JSON is not foundational, actually harmful because it ditches the main idea of web (linking things) for pushing around schemaless hodgepodge data structures...
- nologic01 4y ago
  There is no contradiction. Json simply asserts that the basic exchange blob should be as simple as possible. Linking and schemas and validation and all that stuff are terribly important but a good and simple approach would ADD them when needed, not inflict you with maximum complexity for even the simplest thing. In this respect json-ld has shown one way this can be done but overall its a problem that still asks for a brilliant innovator to solve.
  - throwaway290 4y ago
    Rant warning. There is no complexity in linking, there's nothing simpler than RDF. The whole reason you think it's complex is... drumroll... JSON. It made these bespoke unlinkable structures proliferate and de facto standard. Later JSON-LD tried to bring it back to sanity but because it still has to be JSON (because devs now hate everything else) the end result is complex, inscrutable (contexts?? @reverse???) and usually misunderstood.
    - nologic01 4y ago
      Linking is a conceptual requirement, not a format feature. It is a more complex requirement so it naturally needs more complex semantics to implement. So far so unavoidable. A good design would allow progressively adding semantic complexity iff needed. I think the popularity of json was just because xml violated this very simple design aspect. Unfortunately it is true that linking, metadata, ontologies and all that stuff are required as the ambition of information exchange increases. People who deny that obviously have never worked on any domain that requires significant data exchanges between many independent actors. So now we are stuck with two sub-optimal solutions: a simple but incomplete json approach that must reinvent the wheel without destroying its good bits and a complex and hard to use xml approach that is only loved by software bureaucrats...
nologic01 4y ago
Absolutely loved that interview. So much of current tools we use seem to come from nowhere, but no, there are talented visionaries behind them with very clear ideas of what good looks like and an ability to materialize it. Thankfully some of them succeed. But its a painful slow process as there are not enough of them practical innovators. Big corp and consultants have an amazing ability to keep things stagnant for ages and milk complexity. But the json revolution is definitely unfinished. Standards are important and so many domains have adopted XML already. A practical way to jsonify all these specs would be really useful.
bullen 4y ago
The only feature I lack from JSON (and js) is ordered maps. But XML is not necessarily that bad for many things. I still use this XML editor I made in 2008: http://move.rupy.se/file/logic.html http://move.rupy.se/file/logic.html
- magicalhippo 4y ago
  Assuming you mean maps where the keys are ordered, then that is something JSON Schema could, in theory, specify[1]. However they seem to be against it primarily because it's not JSON (keys are explicitly unordered) and that means some generators might be unable to produce valid output, so interoperability is tricky. [1]: https://github.com/json-schema-org/json-schema-vocabularies/issues/7 https://github.com/json-schema-org/json-schema-vocabularies/...
ttfkam 4y ago
A lot of this reminds me of present day web frameworks. "But you HAVE TO have a virtual DOM. You HAVE TO write all this boilerplate and ship a fat runtime. You HAVE TO manage the caching details to these performance problems and abstraction leaks (looking at you, useMemo!). You have to WRAP AND HIDE the HTML and CSS—which breaks accessibility, so you need to learn all the implementation details about a11y and manually add it back one ARIA role at a time (or just assert it's too hard, so less accommodation)." All this when there are alternatives that are much simpler, so much easier, so much less code, and so much more consistent, folks constantly downplay it. "It CAN'T be used for large projects. You HAVE TO make this large object hierarchy to make web apps. You CAN'T rely on a compiler, because that's magic and no longer 'pure' JavaScript." I especially love the "magic" argument since it completely glosses over the fact that we're providing info to processed sand we hit with lightning to make it think. Magic is already part of the bargain from your first "Hello World." Only now it's not "is that a standard?" It's "everyone's doing it the unnecessarily hard way, and we've invested too much time learning all the ins and outs of the unnecessarily hard way, so if you want a job, you'd better get used to the hard way and suffer as we have."

88 points by ZeljkoS 4y ago

Again? I'll probably listen to this at some point and genuinely like hearing about the origins of JSON from Crockford himself. Apart from that, I'd be surprised if this adds anything to what I've already heard and read ad nauseam, including from Crockford himself. JSON is just a nice, simple serialization format that compresses well and interoperates acceptly with most programming languages. Most importantly with the language that it was designed to be consumed by: Javascript. If one needs better performance or better guarantees about serialized data types, I guess one should use Protobuf, and/or ASN.1 (was that even correctly spelled?) or any of the countless other formats I have no clue about. If you need better schema restrictions and interconnected entities + circular references: use a database, I guess. At least everything XML has done in this space has more or less failed (e.g. SOAP). Well, failed might be too strong a word. But all the work on defining data types and schema seems almost independent of the underlying syntax. Since I've now spent too much time on this uninformed first comment, I will now listen what Douglas Crockford wants to say. But I think I've already read this story in another form a couple of years ago. JSON is really a good example of "worse is better". E.g. an OpenAPI spec with automated client code and TypeScript definitions is not much different from SOAP on the surface level.

wongarsu 4y ago
Two things where imho XML performs well: 1: text with markup. Does not need all the XML machinery, but tags make a lot of sense here. 1b: Arbitrarily nested stuff 2: extensability by third parties without coordination. Namespaces as a central concept help out a lot here, and this use case can even justify some of the more insane features of XML (downloading remote schemas etc) If you don't need either of those things, you won't get much value out of XML. If you do need them, XML can be neat. Certainly easier than trying to allow people to add arbitrary data (with validation) to your protobuf. It's just that these use cases are somewhat rare
- halayli 4y ago
  More importantly, streaming.
- XorNot 4y ago
  I was recently downloading and parsing the US Patent application database, which is emitted in XML. In 2023, Python lxml can't handle multiple documents per file. And no one has any suggestions on how to handle this other then "break on the new document tag". Other libraries didn't look promising in other languages. Then if you try to read it, you need to disable all the DTD stuff because it immediately tries to open other local files on disk...which it can't find. There's some corruption in how it parses DTDs because I had to implement a custom DTD handler in my jupyter-notebook to read it. I could blame the dominant XML processing library of Python, I could blame the lack of attention XML gets, but really? This is just another contribution in a long line of XML not working in some way in every language as soon as you're not using that languages specific XML flavor.
  - dehrmann 4y ago
    I'm curious how Java would handle this. Its XML support is usually very mature. Java and JSON is a different story. There was a window when the only JSON parser Java shipped with was JSON.parse() in its Javascript interpreter.
    - DonHopkins 4y ago
      "Java is a DSL for taking large XML files and converting them to stack traces." -Andrew Back https://www.reddit.com/r/programming/comments/eaqgk/java_is_a_dsl_for_taking_large_xml_files_and/ https://www.reddit.com/r/programming/comments/eaqgk/java_is_... "I Wanna Be <![CDATA[" Sung to the tune of “I Wanna Be Sedated”, with apologies to The Ramones. https://donhopkins.medium.com/i-wanna-be-cdata-3406e14d4f21 https://donhopkins.medium.com/i-wanna-be-cdata-3406e14d4f21
  - nrclark 4y ago
    I haven't had to work with XML all that often. For the few programs I've made that need to speak XML, I've always used the XML module in Python's stdlib. Is that bad practice? What does lxml provide as a benefit?
2h 4y ago
> If one needs better performance or better guarantees about serialized data types, I guess one should use Protobuf ProtoBuf is a horrible format. its not self describing like JSON, when it easily could be, without costing any extra bytes.
- emptysea 4y ago
  aren't there some ambiguities that would require extra data to decipher? like if I have a message inside of a message vs just a bytes field that happens to have a valid protobuf?
  - 2h 4y ago
    Exactly, they could have added an extra wire type to signify messages without costing any extra bytes. Nothing is stopping ProtoBuf from being self describing except incompetence.
paulddraper 4y ago
> At least everything XML has done in this space has more or less failed (e.g. SOAP) Bwhahahaha good one SVG DOCX SAML JSX The legacy of XML incalculable and expanding.
- jeremyjh 4y ago
  XLSX. I'd guess around 80% of the world's code is inside XML.
- throwawaymaths 4y ago
  Svg, docx, and jsx are not in the data interchange business. All three are actually human-consumed documents.
  - recursive 4y ago
    Who's talking about interchange? What data interchanged is not human consumed? Where did those goalposts go?
    - throwawaymaths 4y ago
      JSON is human readable but mostly not human consumed. Something like parquet or protobuf is neither human readable nor human consumed. Xml excels when the content must be human consumed and machine readable.
- deleted 4y ago
  [deleted]
- moritzwarhier 4y ago
  Yeah I had SVG in mind when I wrote that post but I thought of different use cases. Of course every document format can also be called a serialization format. I enjoy JSX as well (and even SOAP!). That's why I weakened the quoted sentence in my comment. When I hear "JSON vs XML" I can't help but mainly think of data payloads in web applications, especially non-static client/server communication. I wouldn't want JSON instead of SVG either :)
DonHopkins 4y ago
It's worth listening to, or at least reading the transcript, just for the Dave Winer burn. (And so much more!) >Adam: That was it. That was the creation of JSON, which everyone is using today. But back then, everyone rejected it. >In some ways it was a marketing problem. On one side, you had Doug, trying to convince customers that they can build interactive applications on the web using JavaScript and this simple thing called JSON. But on the other side, you had XML that had these big companies behind it, IBM, Microsoft, and big consultants. And later they even had some tech influencers like Dave Winer. >Douglas: He’s someone who should have known better. He had a website called scripting.com. His style of scripting came from a clever program that he had written for the Macintosh called Frontier, in which he had a scripting language and an outliner and a word processor and a database, all in one program. And the idea was that you could do virtually anything in Frontier with a little bit of scripting. And he was also one of the big promoters of SOAP, the Simple Object Annoying Protocol. >I don’t remember what the A was, but it might have been atrocious or abominable, I don’t know. >But SOAP was a big deal at the time. They were right in wanting simplicity. They didn’t accomplish it, but they put simplicity in the name as sort of an aspirational thing. And so, when I started showing how JSON works, he was really threatened by that. And on his website, which was well-read at the time, he complained that, “this isn’t even XML. We should find who did this in string them up now”, which was a really ugly thing to say. >Fortunately, nobody listens to Dave Winer, so I’m still here. Dave's done some brilliant influential stuff, which Doug credits and I've written about before, but being annoying is Dave's brand, so Doug's "SOAP" joke is dead on. It's just as fair as referring to Marc Canter's "People Aggregator" as "People Aggravator". https://news.ycombinator.com/item?id=20780928 https://news.ycombinator.com/item?id=20780928 https://news.ycombinator.com/item?id=21170440 https://news.ycombinator.com/item?id=21170440 https://news.ycombinator.com/item?id=16224154 https://news.ycombinator.com/item?id=16224154 https://www.gawker.com/183778/people-aggravator-marc-canter-gets-lonely-spams-for-friends https://www.gawker.com/183778/people-aggravator-marc-canter-...
- moritzwarhier 4y ago
  I did listen to the podcast lateron and was pleasantly surprised by the breadth of topics, although I have to admit I fell asleep halfway through (not because of the contents). Thanks a lot for the money quotes! Casual conversation in English is still sometimes hard to listen to for me without focusing a lot.

deleted 4y ago

[deleted]

theschmed 4y ago

Or you could do what one API I just discovered does: embed escaped XML inside JSON! We truly live in a world

breakfastduck 4y ago
I have worked with a supplier / service who advertised us their brand new shiny REST/JSON based API that we could use to replace the drop a csv in a sftp workflow they offered previously. The API was a post with a single JSON field called data, with a CSV string value. You just know they were asked by management to deliver an API and decided to do quite literally the minimum viable. They had 'no plans' to expand it.
- cgio 4y ago
  How anyone would see that as a MVP when you can make a pretty solid API with Postgrest eludes me.
  - crgwbr 4y ago
    It’s MVP because it (1) is an API and (2) is doing nothing more than writing it’s output to the same sftp server the client used to hit directly. Not debating it’s dumb, but almost by definition enterprise tech ossifies in weird, dumb ways.
- hyperman1 4y ago
  I've seen similar in the wild with XML, except the charset was wrong and escaping was incomplete. So you get the XML header with UTF8 encoding, a root tag, an indented child tag. 3 lines. Then ye olde CSV, with the 5 required entities for ascii escaped, but if you manage to put accented chars in it, they end up in ISO 8859-1. Then 2 lines footer. If you wrote out the file, the header and footer had to be byte for byte identical. Entities encoding non ascii just crashed the parser. Of course, the CSV lines were sensitive to indentation, they just dropped the first 8 chars of each line. Still, XML was officially supported.
icedchai 4y ago

chrisco255 4y ago

This interview is as much about the history of early web development and the serendipitous process by which JSON won out. After reading, it's apparent that JSON could have easily lost and been looked over entirely. Crockford's persistence in evangelizing it even after the company he worked at when it was created went bankrupt led to its eventual adoption.

simplotek 4y ago
> Crockford's persistence in evangelizing it even after the company he worked at when it was created went bankrupt led to its eventual adoption. I don't think this is true at all. The reason that JSON won is stated quite clearly in the interview: you do not need a software stack to handle JSON. This feature sells itself for any software development team who needs to have frontends running in a browser consuming data from a backend. You do not need an evangelist to convince people to use a feature that is ubiquitous, free, extensively tested, and more performance than anything in the market.
- chrisco255 4y ago
  There's plenty of historical examples of sub-optimal standards developing enough traction and network effects that the whole industry is stuck with them for decades. Initially you did need to pull in a library to parse JSON, it wasn't part of the JS language until Crockford pushed for it (via his later position of authority at Yahoo working on YUI). Particularly in the early web, the need for async "liveness" of pages didn't exist, so pages didn't consume backends directly. Everything was handled on refresh or navigation and server rendered as HTML. It was only after AJAX-style web apps took off (post-Gmail, really), that JSON started to get real traction. There were no JSON parsers in C#, Java, Python, etc initially. XML was far more common and SOAP was generally used for cross-endpoint communication.

gnabgib 4y ago

Posted 16 days ago (152 points, 242 comments)[0], 17 days ago (4 points, 1 comment)[1], and twice 19 days ago (1 points)[2], (3 points, 1 comment)[3] [0]: https://news.ycombinator.com/item?id=35467711 https://news.ycombinator.com/item?id=35467711 [1]: https://news.ycombinator.com/item?id=35452861 https://news.ycombinator.com/item?id=35452861 [2]: https://news.ycombinator.com/item?id=35429976 https://news.ycombinator.com/item?id=35429976 [3]: https://news.ycombinator.com/item?id=35427394 https://news.ycombinator.com/item?id=35427394

tough 4y ago
Feels like someone wants XML back
- teleforce 4y ago
  XML ship has sailed, hopefully CUELANG will be widely adopted for a better future.
  - fzeindl 4y ago
    Has it? Is there something akin to XSLT for JSON yet?
    - drzaiusx11 4y ago
      I've made do with .to_xml, use xlst on results, then .to_json. It definably works (with some caveats) but would love to cut out the middleman. I believe xlst 3.0 added explicit json support but I've yet to give it a try: https://www.saxonica.com/papers/xmlprague-2016mhk.pdf https://www.saxonica.com/papers/xmlprague-2016mhk.pdf

jgalt212 4y ago

It was a really good listen for the historical perspective alone. I did not know how much bad blood there was between Dave Winer and Crockford. Winer was a co-creator of XML-RPC, but REST and RPC are two different use cases. I guess they were not so clearly delineated at the time.

denkmoon 4y ago
In practice they're not so delineated today either. How often do you find RPCs labeled "REST-like API". Few "REST-like" APIs actually apply HATEOAS, etc.
- kstrauser 4y ago
  “JSON over HTTP? Yep, we’ve got REST!”
- osigurdson 4y ago
  I think true REST APIs should be called a HATEOS APIs these days to disambiguate. REST just means GET/POST json at this point.
- Shorel 4y ago
  Aptly named HATE-something.

smitty1e 4y ago

Good listen at 1.5 speed.

smitty1e 4y ago

Summary: Crockford is a humble genius and worthy of emulation.

naikrovek 4y ago
humble? lol genius? lol he is certainly not humble and he's no smarter than anyone else. right place, right time. his LACK of humility is what gave him the persistence required.
- smitty1e 4y ago
  While I have no inarguable proof to refute you, and maybe it's easy to sound cool when history has proven one correct, my characterization of "humble genius" is based on DC's low-key presentation within the podcast. At no point did I perceive him to assert any sort of personal superiority. While I use his JSON creation heavily, JavaScript as such is the second least attractive language, after PHP, in my view.
  - naikrovek 4y ago
    at no point did you think that him saying "I was right all along and everyone who opposed me was wrong" is not a message of personal superiority? giving that message is to claim superiority over others, whether it is intentional or not. this podcast is the most humble I have ever heard him, to be fair, but make no mistake; DC is not a humble person. A humble person would not have said "JSON was my/our idea" or "I was right"; a humble person would not be on a podcast talking about themselves at all.

jancsika 4y ago

Side note-- I can just feel the frontend dev's juice running out as they put the finishing touches on the volume slider on that audio player. It suspiciously only updates the attenuation on mouseup. Here's my hypothesis: they initially implemented it the obvious way on mousemove (or whatever similar immediate event). Then they a) noticed the horrible zipper noise that came from doing that, b) felt dread at the idea of plumbing into the webaudio API, c) said "aw fuck it" out loud and d) bound the event to mouseup. Does the dev of that framework post here? If so, am I close on this one? If so I can commiserate from the opposite end-- I spent a long time getting audio stuff like this just right and then fudging text alignment for lack of juice.

tantalor 4y ago
What causes the "horrible zipper noise"?
- fathyb 4y ago
  Any changes in waveforms that are not smooth can cause percussive sounds (that's what percussions are). Playing audio suddenly causes a pop, and changing audio levels abruptly causes a weird zipper sound (the sound of a thousand MIDI drums playing in a few milliseconds). The reasons are very interesting: your brain does a Fourier Transform (sort of) in real time to listen to audio. It allows you to separate two people or two instruments producing sounds at the same time, and abrupt changes will cause this transform to product artefacts. The right way of doing it is to always make progressive changes to the gain level. This property is often used in audio engineering to "add or remove punch" in recordings: you can decrease the attack level to make the gain changes more aggressive and add punch, or lower it to make it smoother, over do it and it sounds muffled. In more concrete words: use the setTargetAtTime WebAudio API.
  - KMag 4y ago
    It's more the cochlea performing some kind of convolution of roughly sinusoidal wavelets with the audio signal, bringing the signal from temoporal to frequency domain, right? Does more neurological post-processing the cochlear output bring it closer to a Fourier transform?

thunderbong 4y ago

This paragraph caught my eye - > I think Object-Oriented Programming was an important advance over procedural programming, which is what we had been doing, but we’re stuck. We’re supposed to change paradigms every generation, and we haven’t. So we’re way behind now. Object-Oriented Programming hit the mainstream in 1980. We’re in the 2020s and we’re still there. We should have moved on.

kagakuninja 4y ago
Functional programming is the next big thing. Ideas from functional programming have gone mainstream, and are found in almost all major languages: lambdas, higher order functions, and immutability / purity. Monads have made some minor inroads (although they get bastardized by OO people, see Java and Javascript).
- silverwind 4y ago
  Yes, but the remnants of OO still linger, both in people's minds and code.
  - klysm 4y ago
    Most importantly in people’s minds. A lot of early educational content is rooted in flawed software philosophy that is carried out into the world
  - usrusr 4y ago
    There are still remnants of ideological OOP use around and they are terrible indeed, but remnants on the language level are perfectly fine as long as they are not used as originally intended. Paraphrasing Crockford, we have found OOP, the good parts. It's used for structuring code now, not for modelling the world. That's a huge difference.
  - DonHopkins 4y ago
    Well if people wouldn't keep brutally deconstructing it [1] then serving it up as a delicious la carte menu [2] of tempting indulgences, maybe object programming will finally die! [1] https://www.info.ucl.ac.be/~pvr/decon.html https://www.info.ucl.ac.be/~pvr/decon.html [2] http://www.paulgraham.com/reesoo.html http://www.paulgraham.com/reesoo.html

paulddraper 4y ago

I find it hilarious whenever someone is treats XML as this archaic thing. When the hot new thing in web dev is... ... ... JavaScript XML (JSX)

afavour 4y ago
I was always under the impression JSX stood for JavaScript Syntax Extension. Whatever it is it certainly isn’t XML.
- recursive 4y ago
  Whatever it is isn't too far from XML.
- paulddraper 4y ago
  > JSX is an XML-like syntax extension to ECMAScript > https://facebook.github.io/jsx/ https://facebook.github.io/jsx/ So yes, it's not XML. It's JavaScript XML.
Shorel 4y ago
> Balanced named tags is a critical syntactic feature of the XML-style notation. This line made me think about the many discussions about Python syntax and the value of having brackets to know when a code block ends and starts. Balanced named tags are another step to make something easier to visually parse, at the cost of having much more text.

osigurdson 4y ago

I like json better but if I'm really honest about it, there isn't really that much difference.

usrusr 4y ago
Sounds like you have missed the "programming in XML" phase that triggered Crockford to do JSON: huge configuration languages, usually with almost good schemas. And nothing in there could ever fail before runtime. It was driven by a deep fear of programming, a desperate hope that maybe you could build something so configurable that you would reach a state of done after which you would never ever have to send the source through a compiler again. In reality however, often the best way to understand a problem in those configuration monsters (or the only way...) was to recompile the non-xml part with some tailored ad-hoc diagnostics. But the idea that the configuration was supposed to change a lot while the code is immutable proved surprisingly robust against ongoing demonstrations of exactly the opposite happening.

yawnxyz 4y ago

You can't spell Complexity without "xml"...

DonHopkins 4y ago

I love Douglas Crockford. >Douglas: Some of the people who were doing JavaScript, were doing it badly and didn’t want to be told that they were doing it badly. There is this notion among second rate programmers that the most important thing they do is express themselves. It’s not making programs that work well and are free of error. That’s way down on the list. What’s much more important is expressing themselves, that, I’m an artist and I express my arts by leaving out semicolons. It’s that kind of thing. So what I was saying did not resonate with that at all. So that audience was not happy to hear it, and JavaScript was just not being used.

DonHopkins 4y ago

>Yahoo Since when did Yahoo lose its "!"?

pkphilip 4y ago

I only wish that JSON had a slightly less verbose format for array types with records with fixed number of columns: example: instead of the following: [{"field1":"value", "field2":"value", "field3":"value"}, {"field1":"value1", "field2":"value2", "field3":"value3"}, {"field1":"value3", "field2":"value3", "field3":"value4"}] If you could have this instead: [{"columns": {"field1", "field2", "field3"}, "rows": [{"value1,"value2","value3"},{"value1,"value2","value3"},{"value1,"value2","value3"}] If json parser could then have the smarts to return the values by position for the various field names even though each row does not include the field names.

RedShift1 4y ago
If you use compression, it's a non-issue.
- IggleSniggle 4y ago
  It’s still an issue for the “readable by humans” use case as it dramatically reduces readability?
5Qn8mNbc2FNCiVV 4y ago
It's JavaScript Object Notation. An array of objects looks like that, if you want to have another data structure you can do it and have the accompanying code to consume it. The only thing holding you back from doing that is 1kb of JS as the second argument to JSON.parse

throwaway290 4y ago

> JSON is foundational stuff for the web JSON is not foundational, actually harmful because it ditches the main idea of web (linking things) for pushing around schemaless hodgepodge data structures...

nologic01 4y ago
There is no contradiction. Json simply asserts that the basic exchange blob should be as simple as possible. Linking and schemas and validation and all that stuff are terribly important but a good and simple approach would ADD them when needed, not inflict you with maximum complexity for even the simplest thing. In this respect json-ld has shown one way this can be done but overall its a problem that still asks for a brilliant innovator to solve.
- throwaway290 4y ago
  Rant warning. There is no complexity in linking, there's nothing simpler than RDF. The whole reason you think it's complex is... drumroll... JSON. It made these bespoke unlinkable structures proliferate and de facto standard. Later JSON-LD tried to bring it back to sanity but because it still has to be JSON (because devs now hate everything else) the end result is complex, inscrutable (contexts?? @reverse???) and usually misunderstood.
  - nologic01 4y ago
    Linking is a conceptual requirement, not a format feature. It is a more complex requirement so it naturally needs more complex semantics to implement. So far so unavoidable. A good design would allow progressively adding semantic complexity iff needed. I think the popularity of json was just because xml violated this very simple design aspect. Unfortunately it is true that linking, metadata, ontologies and all that stuff are required as the ambition of information exchange increases. People who deny that obviously have never worked on any domain that requires significant data exchanges between many independent actors. So now we are stuck with two sub-optimal solutions: a simple but incomplete json approach that must reinvent the wheel without destroying its good bits and a complex and hard to use xml approach that is only loved by software bureaucrats...

nologic01 4y ago

Absolutely loved that interview. So much of current tools we use seem to come from nowhere, but no, there are talented visionaries behind them with very clear ideas of what good looks like and an ability to materialize it. Thankfully some of them succeed. But its a painful slow process as there are not enough of them practical innovators. Big corp and consultants have an amazing ability to keep things stagnant for ages and milk complexity. But the json revolution is definitely unfinished. Standards are important and so many domains have adopted XML already. A practical way to jsonify all these specs would be really useful.

bullen 4y ago

The only feature I lack from JSON (and js) is ordered maps. But XML is not necessarily that bad for many things. I still use this XML editor I made in 2008: http://move.rupy.se/file/logic.html http://move.rupy.se/file/logic.html

magicalhippo 4y ago
Assuming you mean maps where the keys are ordered, then that is something JSON Schema could, in theory, specify[1]. However they seem to be against it primarily because it's not JSON (keys are explicitly unordered) and that means some generators might be unable to produce valid output, so interoperability is tricky. [1]: https://github.com/json-schema-org/json-schema-vocabularies/issues/7 https://github.com/json-schema-org/json-schema-vocabularies/...

ttfkam 4y ago

A lot of this reminds me of present day web frameworks. "But you HAVE TO have a virtual DOM. You HAVE TO write all this boilerplate and ship a fat runtime. You HAVE TO manage the caching details to these performance problems and abstraction leaks (looking at you, useMemo!). You have to WRAP AND HIDE the HTML and CSS—which breaks accessibility, so you need to learn all the implementation details about a11y and manually add it back one ARIA role at a time (or just assert it's too hard, so less accommodation)." All this when there are alternatives that are much simpler, so much easier, so much less code, and so much more consistent, folks constantly downplay it. "It CAN'T be used for large projects. You HAVE TO make this large object hierarchy to make web apps. You CAN'T rely on a compiler, because that's magic and no longer 'pure' JavaScript." I especially love the "magic" argument since it completely glosses over the fact that we're providing info to processed sand we hit with lightning to make it think. Magic is already part of the bargain from your first "Hello World." Only now it's not "is that a standard?" It's "everyone's doing it the unnecessarily hard way, and we've invested too much time learning all the ins and outs of the unnecessarily hard way, so if you want a job, you'd better get used to the hard way and suffer as we have."