3 ms·
"We’ve got a super-fast in-house graph storage system that makes it possible to do interesting stuff with graphs quickly, notably figure out which pages are rel
by randomwalker 18y ago
"We’ve got a super-fast in-house graph storage system that makes it possible to do interesting stuff with graphs quickly, notably figure out which pages are related."
I'm working with large graphs and I find that using a relational database as a backend is horribly slow if you want to run somewhat complex graph algorithms. Looks like everyone who does this ends up developing an in-house system. Anyone know if there's a library out there for doing this sort of thing? The order of magnitude I'm talking about is ~10^8 nodes, ~10^9 edges.
- fizx 18y agoI think your best bet is an RDF tool like Jena or Sesame. 10^9 is pushing these engines though. There's an HBase-SPARQL lib, but that's gotta be two years away from stable. For real world problems, I tend to write custom code that's Java NIO heavy. Try to pack the data as efficiently as possible, and minimize disk seeks.
- wheels 18y agoThe tools that you mention are honestly not just one, but a couple orders of magnitude too slow to do interesting applications on at this scale.