3 ms·
There is so much useful and well-organized information in RDF format out there that is completely being unused. Check out: http://wiki.dbpedia.org/Datasets http
by chime 17y ago
There is so much useful and well-organized information in RDF format out there that is completely being unused. Check out: http://wiki.dbpedia.org/Datasets http://wiki.dbpedia.org/Datasets - this includes almost the entire Wikipedia in a nice, organized way. Using Sparql/Snorql ( http://dbpedia.org/snorql/ http://dbpedia.org/snorql/ ) you can actually find everything that Bart Simpson wrote on the chalkboard during the opening sequence in season 12: http://www.snee.com/bobdc.blog/2007/11/querying-dbpedia.html http://www.snee.com/bobdc.blog/2007/11/querying-dbpedia.html
However, searching through RDF is a pain and not a small task. The tools available to manage 50GB worth of RDF data are very basic and inefficient. I used a MySQL-based RDF store and even on my extremely powerful server, doing a simple query choked the machine. I'm hoping with a better data store, we can now manage RDF data better and hopefully search it better. Think of how much useful information there is on Wikipedia, Powerset.com etc. I don't mean just the text, but rather the extremely well-organized human-edited information boxes on the right that contain categories, hierarchical data, and specifications. Using RDF, all of that could be instantly queryable.
I would love to write an app that queries a Google RDF store for local listings or map information. APIs are flat and tabular. I can only query Amazon for specific search words in items. Right now, I can't say with one query: "show me all ratings for every book that was written by the authors who currently have a book in the top 100 books list." RDF makes that possible. The problem right now is that RDF stores suck and searching through them isn't easy for the average developer. Here's hoping Apache makes it possible and scalable.
- jerf 17y ago"The problem right now is that RDF stores suck and searching through them isn't easy for the average developer." The question is, can there be a such thing as a non-sucky RDF store, though? RDF is a directed graph, essentially the single most general data structure possible. That means it can represent anything, awesome!... that also means, no optimizations that take advantage of the structure of the data are possible. It is not clear to me that the combination of "no effective optimizations" and "enormous internet-scale database" actually has a solution. (It certainly doesn't have an easy solution, you'd have to do something very sophisticated to extract structure back out and take advantage of it, while admitting the existence of arbitrary exceptions to your clever scheme. Even if possible, this is a recipe for an endless progression of the hardest sorts of bugs.)
- elblanco 17y ago> "show me all ratings for every book that was written by the authors who currently have a book in the top 100 books list." Why not? That seems like a pretty basic SQL query to me, with one nested query. I think we even did something almost exactly like that in my undergrad dB database management class in the first few weeks. something like (sorry, I'm the world's worst SQL guy) SELECT title, ratings FROM books WHERE author IN (SELECT author FROM top100books); If I just had one table with the sales figures, titles and ratings in it, I could even derive the answer in one query also. I'm sure my syntax is all wrong and messed up, but that's not the point. The point is most of the notional "only RDF could make this possible!" examples I've heard are pretty much completely doable in one or two queries in SQL, and not even particularly complicated ones at that.
- epochwolf 17y ago> "show me all ratings for every book that was written by the authors who currently have a book in the top 100 books list." Why not? That seems like a pretty basic SQL query to me, with one nested query. I think we even did something almost exactly like that in my undergrad dB database management class in the first few weeks. That's more like (sorry, I'm bad at sql too) SELECT ratings.*, book.name, author.name FROM ratings JOIN books ON ratings.book_id = book.id JOIN authors ON books.author_id == author.id WHERE author.id IN ( SELECT author_id FROM books JOIN ratings ON book.id = book_id ORDER ratings.score ASC LIMIT 100 ) You better hope that sucker uses indexes or it's going to be a few minutes. And you better cache that because you don't want that running every time someone hits the recommended reads. SQL can be pretty scary for this stuff. It would be even worse if you allows a many to many relationship with books and authors.
- elblanco 17y agoLike I said, I'm the world's worst SQL guy. But the good news is that I haven't invalidated my point by my sloppy syntax. So this raises two questions... 1) Who doesn't use indexes these days? Even light solutions like SQLite support them. 2) The only real remaining advantage to RDF is that the data resources can sit on the end of URLs if I remember. I think this is mainly because nobody has put much effort into dangling indexed relational tables at the end of a URL. There's nothing that says I couldn't have SELECT ratings.*, book.name, author.name FROM http://someplace.com/booklists.ratings AS ratings JOIN http://someplaceelse.net/booksales.books AS book ON ratings.book_id = book.id JOIN http://yetsomeotherplace.org/publisherdata.authors AS authors ON books.author_id == author.id WHERE author.id IN ( SELECT author_id FROM books JOIN ratings ON book.id = book_id ORDER ratings.score ASC LIMIT 100 ) Or some such...sure it's sloppy and it's slowish, but aren't those the same problems as with RDF? And yet we're still using SQL to do this.