6 ms·

Alright, in the flamebait fashion, I'll bite back :) > Yeah, and how often do you merge two data formats like that into one data format in a way that doesn't r

• deleted 10y ago
  [deleted]
• qwertyuiop924 10y ago
  Okay, back to me: >Actually, quite a lot in the past! Back in 2009 I did some XProc pipelining of messages. These pipelines were a bit like reactive streams, which were (mostly) agnostic of the contents. This allowed me to combine, dissect and route streams of data in an intuitive way. Huh. So like this: |xmlstream|->|transformer|->|xmlstream| Pretty slick. So the namespacing allowed you to add new tags, without worrying about tripping over the old ones? Cool, but the types of transforms you can do without knowing the internals of the XML you're transforming are fairly limited, and because JSON's objects don't mandate an app-wide wide meaning for a key - the closest thing JSON has to XML tags, you can just attach the new data to a new dict, and the problem solves itself. If you're merging objects, and each gives a different value for a key, than you can set up either an array or an object to hold both, or just send along both objects, wrapped in an array/object like before: in essence, by JSON'S semantics, each object is its own namespace. >Reference, please? Even if I subscribe to one definition of 'wire protocol' on the internet (there are many), I don't think it creates a meaningful distinction between XML and JSON. References, I can give. json.org, first paragraph: JSON (JavaScript Object Notation) is a lightweight data-interchange format. I apologize for being unclear: Data Interchange format is what I meant. XML was not intended to be a generic data-interchange format: It, like HTML, SGML, and GML before it, were designed for DOCUMENT markup: Human-readable, structured, semantic DOCUMENTS. It has since been pressed into service as a data-interchange format, and it's a testament to how well it was designed that it works as well as it does for that, but its verbosity and general format and layout make it ill-suited to the purpose. JSON was designed for data interchange: I said wire protocol, as Data Interchange is often about sending data between applications on a network, which is what a wire protocol is for. Hopefully some of that answers your question. >Actually, it is, and it isn't. Yes, there are very few primitives (strings, booleans, numbers, arrays, objects), but this also causes important limitations. For example, it is rather cumbersome and unspecified to transfer binary data in a JSON document (base64 encoding). Another thing: how easy is it to parse a streaming JSON document in Javascript? Is it specced to transfer binary data in XML? First I'd heard of it. Base64, uuencode, hex, or raw numbers, there are plenty of ways to encode binary data in JSON, and if you're using any system that has reserved characters (like CDATA in XML, if that's what you're thinking of), than you have to do this sort of encoding somehow. Besides, you could always send the json as a header, and have the app get the binary data from a different endpoint. Although you may want Base64 to avoid the roundtrips... As for parsing streaming JSON, I don't know of there are any libraries for it, but the implementation should be very simple: Like XML, JSON is a tree, so parser state can be represented as a stack: You see a {, you're now in an object. A [, you're in an array. A , indicates adding a new value to the current array, or a new k/v pair to the current dict. What each character means is deterministic, given what came before, so you can construct JSON as the data comes in, and provide access to each value as it becomes ready. Although, given most JS implementations' multithreading limitations, all this really does is ensure that you don't have to have the entirety of the data in memory before you start parsing. Which is a good idea...
  • edejong 10y ago
    wrt the XProc pipelining: it's been some time, but I recall various possible transforming and matching steps, such as a conditional stream, transformers, reduction steps of multiple streams and others. This could then be combined with XPath, XQuery, XSLT and even SOAP requests. The problem of XProc was similar to the rest of the XML era: it required too much of the implementer to understand. Also, good programmer tooling, such as graph editors for the pipelines were missing. Perhaps this is slightly similar to functional programming and category theory nowadays. The ideas are sound, but it produces too much study and too little profit. Also, to properly work with category theory in programming, it would be very nice to have some graphical tools to view the transformations applied to your code. Now for the flamebait-y parts. Yes, XML has loads of archaic SGML syntax bits, DTD built-in (hopeless for small parsers) and the attribute/sub-element divide has never been completely solved. But it can be argued that JSON had a similar fate: it literally descended as a subset of ECMAscript. This also explains the lack of separation between integers and floats. But I agree, XML is definitely not the best data-interchange format, but neither is JSON. Some LISPy syntax would be my preference for data-interchange if it needs to be readable. But I'm trying to argue that this doesn't really matter. The XML era is mostly over, and I'd say we should try to learn from 'the good parts'. I conflated XML with the XML Schema datatypes, and I shouldn't have, but it has been some time since I last seriously worked with XML. Also, we should consider the whole ecosystem, not just structure. XML Schema actually does spec binary data (http://www.datypic.com/sc/xsd/t-xsd_hexBinary.html http://www.datypic.com/sc/xsd/t-xsd_hexBinary.html). With regard to round-trips: as a general rule it might not hold if you ask me. It implies an origin and even state! Maybe the sender cannot cache your binary data for a round-trip (memory, legal, latency, security constraints all play a role here). Personally, I like RESTful for simple systems, but for more involved architectures, message passing is much more scalable and easier to distribute. > As for parsing streaming JSON, I don't know of there are any libraries for it, but the implementation should be very simple ... Yes, a novice programmer should be able to write it in an hour. But the weird thing is, of all our libraries and all our frameworks (browser-side), none of them do streaming. Ok, I guess we should use websockets with JSON-encoded events for this, but still. But hey, there are so many metrics with which one can evaluate a data-interchange format. (Recently we did a survey of binary data-interchange formats and found around 25 different criteria... and we were not really being thorough).