4 ms·
Look at Facebook, it is one giant social graph, with the problem of how to find the average distance from anyone to anyone. You can simulate a graph as an edge
by nl 11y ago
Look at Facebook, it is one giant social graph, with the problem of how to find the average distance from anyone to anyone. You can simulate a graph as an edge matrix, and a connectivity matrix in an array-based system, and you model graphs in a table system, or you build a special-purpose engine to implement the graph directly. All three are being prototyped and commercialized, and the jury is out whether there is room for a new graph engine or if one of the other technologies would be good enough.
This.
So I'm dealing with this problem. There is nothing out there.
Neo4J doesn't really do in-graph processing[1]. BlazeFB/OrientDB/RDF Stores all are similar to Neo4J
Pregel/GraphX/Giraph are graph processing engines, but lack property stores.
I want a single system that does both. I want to run PageRank (etc) and query-by-property on the same system.
Titan was promising, but they stopped working on it when they were bought.
I'm surprised no one is fixing this.
[1] http://neo4j.com/blog/categorical-pagerank-using-neo4j-apache-spark/ http://neo4j.com/blog/categorical-pagerank-using-neo4j-apach... note this bit: "I can scale each Apache Spark node to perform parallel PageRank jobs on independent and isolated processes all consuming a Hadoop HDFS file system where the Neo4j subgraphs are exported to." (ie, Spark runs against HDFS, not Neo4j)
- mooreds 11y agoDoes this get you closer to where you need to be? http://www.kennybastani.com/2015/03/spark-neo4j-tutorial-docker.html http://www.kennybastani.com/2015/03/spark-neo4j-tutorial-doc...
- nl 11y agoNo. My reference above is about that technique.
- herewego 11y agoShameless self promotion: My company is working on an open source distributed hypergraph database that tackles this problem, called PatternSpace. Written in C++14, uses Paxos, Cap'n Proto, lots of mechanical sympathy, and is queryable/traversable via subgraph isomorphism among other methods.
- dunkelheit 11y agoCould you provide some pointers to your work? I googled a bit and searched on github and could not find anything.
- jerven 11y agoHave a look at blazegraph it can do both sparql and gremlin. Has support for runninh on clustered gpu. Was selcted by wikidata when titan was bought.
- nl 11y agoBlazeFB (wtf?) above was supposed to be BlazeGraph.
- deleted 11y ago[deleted]