6 ms·
plaintext + gzip is preferable to binary. The early adopters of XML saw their choices as semistructured text vs nice parseable XML. The third choice, creating
by specialist 15y ago
plaintext + gzip is preferable to binary.
The early adopters of XML saw their choices as semistructured text vs nice parseable XML. The third choice, creating grammars, was ignored.
Grammars for most configuration or data transfer or protocols are trivial. Certainly since ANTLR 2.x. Much more trivial than any equivalent XML-based parse and validation tool stack.
FWIW, ASN.1 is worse that XML. Not a defense of XML; all uses of XML are incorrect.
For my own work, I use a format descended from VRML that I call ARON (A Righteous Object Notation).
It concisely describes groves (trees annotated with key/values) and supports most commonly used datatypes. So it's a bit more concise that JSON or YAML and a lot more strongly typed.
Here's an example test file:
http://code.google.com/p/aron/source/browse/trunk/test/cronk/test1.aron http://code.google.com/p/aron/source/browse/trunk/test/cronk...
I use ARON for all my own projects, as you can see, it's not really polished enough for others (yet). As this example shows, I mostly use it to loft Java object graphs. I haven't reimplemented VRML's prototyping (DEF / USE) functionality in this branch (yet).
- strags 15y ago>>> plaintext + gzip is preferable to binary. I'm not sure I agree with you on that. You're imposing a load of extra CPU (and, I suspect some bandwidth) overhead where it's not necessary for all except the most infrequent cases, and you're inheriting all the weaknesses of text as a data format. Plus, since your data is now gzipped, it's no longer human-readable on the wire. In order to read it, you need to pipe it through a decoder (gunzip) - why not use a sensible binary protocol, and pipe it through the decoder for that?
- guard-of-terra 15y agoIt is human readable if you use firebug or vim. And that's what you use.
- derleth 15y ago> In order to read it, you need to pipe it through a decoder (gunzip) - why not use a sensible binary protocol, and pipe it through the decoder for that? I don't have your decoder. I have gunzip. gunzip is not threatened by a patent. gunzip doesn't cause a Drama Meltdown. gunzip won't be a proven attack vector for remote execution exploits. gunzip does not require a contract in my hand or money in my bank. While your decoder is being debugged, gunzip will be live. (To the tune of "The Revolution Will Not Be Televised")
- strags 15y agoOh, I'm just advocating using a standard binary serialization format like MessagePack that is far more efficient, can easily represent binary data, and has a far more obvious mapping to runtime data structures.
- seanp2k2 15y ago>"standard binary serialization format" Mmmm, yes, good luck with that :)
- strags 15y agoA man can dream.
- stock_toaster 15y agoIn most parsing comparisons I have seen, tnetstrings[1] is probably one of the faster textual formats to parse (and rather "safe" too). I have started using tnetstring (python module) for some backend message formats where I commonly use json, and it has been pretty great so far. (esp with gzip, snappy, bzip, lzma, or whatever compression fits your needs/tradeoffs) [1]: https://github.com/j2labs/cerealization https://github.com/j2labs/cerealization
- stock_toaster 15y agoIn most parsing comparisons I have seen, tnetstrings[1] is probably one of the faster textual formats to parse (and rather "safe" too). I have started using tnetstring[2] (python module) for some backend message formats where I commonly use json, and it has been pretty great so far. (esp with gzip, snappy, bzip, lzma, or whatever compression fits your needs/tradeoffs) [1]: http://tnetstrings.org/ http://tnetstrings.org/ [2]: http://pypi.python.org/pypi/tnetstring http://pypi.python.org/pypi/tnetstring [3]: https://github.com/j2labs/cerealization https://github.com/j2labs/cerealization
- guard-of-terra 15y agoXML has XPath. JSON and YAML do not have xpath. They have those lame language-specific constructs that aren't recursive, aren't traversable and throw null pointer errors when they can't match. You can format XML with xmllint and query it with xmlstarlet. You can't do that for json - good luck if you've got unformatted json. Tooling is everything.
- specialist 15y agoI added lightweight path expressions to LOX (Lightweight Objects for XML). It's a modern XML object model, that's easy to inspect, with built in support for xpath-like expressions (globbing, really), without all the suckage. http://code.google.com/p/lox/ http://code.google.com/p/lox/ Here's some example of path expressions. http://code.google.com/p/lox/source/browse/trunk/test/lox/test/Expression.java http://code.google.com/p/lox/source/browse/trunk/test/lox/te... Works fantastically. I use LOX for all my own XML work. My ARON project doesn't support path expressions in the same way. ARON's grammar only supports drill down dot notation, like "parent.child.grandchild". And, thus far, I've only used ARON to loft Java object graphs, so I haven't needed an object model with path expressions. I'd love to see a high level, statically typed language with built-in path expression. Groovy's GPath is closest to the mark that I've seen. (I really need to polish these open source side projects. And publicize them.)
- guard-of-terra 15y agoMaking your own is cool, but where's the value added compared to XPath?
- specialist 15y agoThanks! It was a lot of fun and proven very useful. Biggest reason is conciseness (less code), followed by ease of debugging. When using JXPath or Jaxen, you have to use contexts. Pseudo code (from memory): Node n = new Node( "ugh" ); // add some children here JXPathContext c = new JXPathContext( n ); NodeList list = c.eval( "child" ); for( int i = 0; i < list.length(); i++ ) { Node m = (Node) list.get( i ); ... } Whereas LOX does it like this: Element e = new Element( "ugh" ); // Add some children here for( Element e : e.find( "child" )) { ... } I do a lot of ETL work (XML -> SQL). The ability to debug (interactive inspection) speeds development. I'm sure you've tried to debug XPath expressions. Not easy. Happily, all of the LOX's object implement toString() method to render their XML content. And the evaluation of path expressions, while not easy, is feasible. Whereas with JXPath or Jaxen, it's damned near impossible. (I've written a few object adapters for JXPath; getting them right is black magic.)
- 6ren 15y agoI think there are two issues with XML/JSON/etc vs. roll-your-own format/grammar. The first issue is the difficulty of creating it, which you describe as "trivial". I think it's an "easy once you know how", but not easy beforehand, not easy for everyone, and not every aspect of it is easy even then. Another way of looking at this is that designing and writing and testing and debugging and rewriting and documenting a grammar is always going to be more work than using pre-existing code. For example, although you've put in enough work to be able to use ARON in several projects, it's not ready for others. IMHO it's a non-trivial undertaking. (BTW: does ANTLR 2.x automatically take care of left-recursive grammars?) My experience with parsing is that there be gotchas; this is one reason I'm very impressed with the principles behind XML Schema. (It's a shame it's so soul-destroyingly horrific to actually use. Also XML namespaces: righteous concept, diabolical realization.) The second issue is familiarity/standardization vs. specific-use formats (a kind of DSL), purely in terms of ease-of-use and adoption. I think familiar formats really are easier to read - your eye and brain have internalized short-cuts for interpreting them, so you can quickly work on them. Similar to your finger-memory for your editor. An alternative must be significantly better to overcome this unfair advantage of the incumbent. However, counter-example: for both these reasons (the first is a kind of mechanized version of the second), I thought it would be impossible to displace XML, much as ASCII seemed impossible to displace (there used to be competing character encodings; the present standard, unicode, can be seen as a superset of ASCII). Although JSON seemed wonderfully better (I first saw it in the ActionScript version of ECMAscript), I thought it couldn't overcome XML's incumbency. Yet, JSON is making inroads, and not just in web-clients, but in APIs. It's interesting that JSON lacks analogs to XML Schema, XSLT, XPath; although proposals have been made, they aren't adopted. I wonder if this is because people are now figuring out how to do without these extras? Or if, when they do need them, they just use the XML stack... I think new formats/grammars have the best chance of adoption in niches, when they are so specialized to the task that their benefits far outweigh the advantages of the standardized incumbent - in that particular use-case. For example, most programming languages define their own grammars e.g. python, ruby, java, etc. BTW: in your other comments with code, just indent each line by two spaces for nicer formating. :-)
- specialist 15y ago(Love your metaphors/adjectives.) Grammars aren't easy, but not currently no harder than XSD. Further, DSLs are all the rage. Personally, I prefer to use a pro tool like ANTLR to some in language voodoo with Scale (or some such). LALR (lex/yacc) kicked my ass. Just could not understand it. LL(k) (ANTLR 2.x, JavaCC) was a struggle. But I slogged thru it. LL(*) (ANTLR 3.x) is a pleasure to work with. I really enjoy it. For example, the delta between SQL's (idealized) BNF and my SQL ANTLR grammar is very small. ANTLR 4.x, currently in progress, will do left recursion, which I haven't played with yet. I'm not too worried about adoption. My tools make me more productive. If my competitors choose to use rocks for hammering, I'm okay with it. Lastly, I indent with tabs. Old habits die hard. :)