3 ms·

Which database do you use to store your RDF data, which supports SPARQL, and how does it perform? > I just use raw RDF, and defined my own vocabularies for ev

by mhitza 3y ago

Which database do you use to store your RDF data, which supports SPARQL, and how does it perform? > I just use raw RDF, and defined my own vocabularies for everything, including structure. I think this is best approach. The ontology part was more of a hindrance for me way back in 2010's when I was experimenting with semantic web technologies (using dbpedia as a source of my data) and I really hard tried to avoid going of the beaten path (no matter how flaky it seemed) as a junior level developer.

whartung 3y ago
I'm using Apache Jena and Java, and using TDB. Performance? I have nothing to compare it too. I can't complain. I know whenever I saw info about triple stores in the past, they only seem to crow about was how fast it takes to import things. "Eleventy trillion triples per bleem!" I guess nobody ever actually queries the data, they just store it. I routinely export the model to a file, and that takes seconds (<10), using the N3 format (RDF/XML takes a very long time). I export the model to make sure my changes are reflecting properly. The resulting file is 176MB. If I read that into a new TDB instance, I can load it in 25s. Since I've been importing from a SQLite master, the resulting TDB data is roughly the same size as the SQLite DB file. My import from SQLite takes longer than 25s, and that's just data shoving from the SQL data to the RDF. I'm sure I'm the bottleneck in that case, I probably commit too much for one thing. As for queries, well, it either can find it or it can't. It's either trivially indexed (I assume each of the properties of the triple are indexed), or it table scans. Internally when you do a query from Java, you basically set the base net you want to throw (you've only got 3 values to work with) and iterate through to filter it. When I did my "select count" query to count all the triples, that took a beat or two to be sure as it hoovered the entirety of the model and cursored through it. I have not done any crazy SPARQL queries (I can barely spell SPARQL), so I don't know what kind of decisions is makes, but, in the end, there's really only a few ways you can actually query a triple store. Now, I have a recent Intel iMac I'm running this on, so that may well impact things as well. I have no idea how much memory I'm using, it hasn't been a problem. I've done no tuning whatsoever, I honestly don't know what tuning is available. I do not foresee my dataset growing much more, so TDB is "fast enough" for my purposes. All told I'm pretty happy with everything.

by mhitza 3y ago

whartung 3y ago

I'm using Apache Jena and Java, and using TDB. Performance? I have nothing to compare it too. I can't complain. I know whenever I saw info about triple stores in the past, they only seem to crow about was how fast it takes to import things. "Eleventy trillion triples per bleem!" I guess nobody ever actually queries the data, they just store it. I routinely export the model to a file, and that takes seconds (<10), using the N3 format (RDF/XML takes a very long time). I export the model to make sure my changes are reflecting properly. The resulting file is 176MB. If I read that into a new TDB instance, I can load it in 25s. Since I've been importing from a SQLite master, the resulting TDB data is roughly the same size as the SQLite DB file. My import from SQLite takes longer than 25s, and that's just data shoving from the SQL data to the RDF. I'm sure I'm the bottleneck in that case, I probably commit too much for one thing. As for queries, well, it either can find it or it can't. It's either trivially indexed (I assume each of the properties of the triple are indexed), or it table scans. Internally when you do a query from Java, you basically set the base net you want to throw (you've only got 3 values to work with) and iterate through to filter it. When I did my "select count" query to count all the triples, that took a beat or two to be sure as it hoovered the entirety of the model and cursored through it. I have not done any crazy SPARQL queries (I can barely spell SPARQL), so I don't know what kind of decisions is makes, but, in the end, there's really only a few ways you can actually query a triple store. Now, I have a recent Intel iMac I'm running this on, so that may well impact things as well. I have no idea how much memory I'm using, it hasn't been a problem. I've done no tuning whatsoever, I honestly don't know what tuning is available. I do not foresee my dataset growing much more, so TDB is "fast enough" for my purposes. All told I'm pretty happy with everything.