4 ms·
XML databases actually got to be amazing, if that was his prediction he would have been totally right. The fact is that they just got amazing a little bit too l
by tinco 11y ago
XML databases actually got to be amazing, if that was his prediction he would have been totally right. The fact is that they just got amazing a little bit too late. No one is going to consider an XML database because XML got a bad reputation (and rightly so).
But XML database technology actually has nothing to do with XML. The idea is that you store nested data nodes. Imagine you have a mongodb collection, and in it you have documents, and each of those documents would also be a mongodb collection in which you could insert more documents. You can build full graphs like that, and if you allow references to nodes those graphs can be cyclic too. Pretty powerful right?
Now you might say yeah of course, but that's got to be crazy slow! And at the time Yegge made his prediction you would have been right. But then academics came into the scene, they developed data structures and algorithms that allow you to store these recursively nested documents in column stores (think SQL table with 2 columns, a primary key and a piece of data) in such a genius way that you can make whatever query you can think of (recursive, joins, whatever) and execute them almost as fast as any indexed search query.
That magic happens, and is implemented for example in MonetDB/XQuery which sadly got abandoned a few years ago.
I still think if a crazy person sets off to go build a database company (crazy people like that seem to be quite popular in SV these days) they could make a lot of money forking MonetDB/XQuery, and search-replaceing every instance of X with J and every instance of XML with JSON. Crazier still, you can even let MonetDB/XQuery results be formatted as json, even before MongoDB became popular I think.
- andrewflnr 11y agoCould you go into a bit more detail about the data structures and algorithms used in doing recursive queries on nested data in a column store? Pointers to relevant papers/documentation would be great.
- tinco 11y agoThis paper summarises the MonetDB/XQuery work, I'm on my phone and couldn't find a free PDF but I bet there's one somewhere. I read this paper while I was an undergraduate, it's very accessible. http://dl.acm.org/citation.cfm?id=1142527 http://dl.acm.org/citation.cfm?id=1142527
- DougN7 11y agoIsn't that just the Object databases from 15 years ago? I remember using one from Versant that seems to be what you're describing.
- fsiefken 11y agoOpenLink Virtuoso implements a lot of this functionality (xml storage, column store, graphs, rdf, json), optimized for speed and open source. I know because I was lucky enough to have had conversations about it with the guy responsible for it, Orri Erling http://sites.computer.org/debull/A12mar/p3.pdf http://sites.computer.org/debull/A12mar/p3.pdf