7 ms·

What exactly is this? The GitHub page speaks of different backends, and those appear to just be databases or key-value stores in themselves (e.g, Postgres and B

What exactly is this? The GitHub page speaks of different backends, and those appear to just be databases or key-value stores in themselves (e.g, Postgres and Bolt). Is Cayley basically a query rewritter, that is it has some tables in the backend and when queried, Cayley then goes to the "real" (for lack of a better word) database? Cayley's query language might be more full featured, but it isn't a storage mechanism in itself? There are two things from that: 1. There is no way for Cayley to take the graph structure of the data into account when laying it out on disk or when executing the query. Is this the long-term decision, or is this just a stop-gap until a storage mechanism can be done? 2. This would seem to imply that the abstraction layer from Cayley to the backend storage would be relatively slim. How difficult is it to add another storage driver for another SQL database or for one with a custom query language? Another thing I noticed: > query -- films starring X and Y -- takes ~150ms Even on two year old hardware that seems dog slow - less than 7 queries a second - for a very simple query.

perfmode 10y ago
You can't derive rate from latency alone. The query might be IO bound. If that's the case, you can run queries concurrently.
- jnordwick 10y ago
  I tend to care more about latency than throughput. FedEx has the best throughput in the world if you can wait a couple days. True though. I'm guessing with a latency number like that, the throughput is pretty bad too.
  - ashwin95r 10y ago
    Exactly what https://dgraph.io https://dgraph.io is built for. Low latency, high throughput.
    - jnordwick 10y ago
      Looks interesting. That semi-declarative query language is hideous though. Reminds me of attempts to make XML into a programming language.
      - mrjn 10y ago
        (author of Dgraph) It's based on GraphQL, which is definitely not a hack like XML => language and currently catching on faster than a forest fire. Dgraph's derivation, GraphQL+- (for lack of better name) is a lot more powerful than Cypher or Gremlin. Both of the latter allow only returning lists of results; while GraphQL+- returns back an entire subgraph. Thus, all the relationships are maintained. It also allows expressing complex joins using a function like variable blocks; feeding results from one into another like you'd do in any popular language. You can try running some complex queries with GraphQL+-, it might change your view. http://play.dgraph.io/ http://play.dgraph.io/
        jnordwick 10y ago
        Cool af Edit: I still think it looks hideous. Def try to figure it out more tomorrow.
dimfeld 10y ago
Cayley's graph data layout is most similar to a Hexastore-style [1] triple store, though IIRC it doesn't do the full six-way index that the original Hexastore paper describes. The Redis page on secondary indexing [2] has a great quick intro to what this actually entails (search the page for Hexastore). As you might guess from the Redis link, this style of graph lends itself well to KV stores, so the answer to your question #1 might be that it's a long-term decision, but the style of graph is really designed for a KV store anyway. But I haven't discussed this at all with the Cayley devs so I can't actually speak for them. I'm using it with the BoltDB backend and have been pleased with the performance overall. I haven't looked at the backends for more complex databases like Postgres in detail, but it does appear that the backend interface has potential for predicate pushdown as well. The repository's graph directory [3] contains the various backends if you want to check it out. Overall it doesn't look very difficult to add another backend type, but I haven't tried it yet. Looking at the existing SQL backend, it appears to already support MySQL, PostgreSQL, and CockroachDB (but I've tried none of these with Cayley). [1] http://www.vldb.org/pvldb/1/1453965.pdf http://www.vldb.org/pvldb/1/1453965.pdf [2] https://redis.io/topics/indexes https://redis.io/topics/indexes [3] https://github.com/cayleygraph/cayley/tree/master/graph https://github.com/cayleygraph/cayley/tree/master/graph
- itamarhaber 10y ago
  Speaking of which, take a look at this Redis module that marries Hexastore and neo4j-like queries: https://github.com/RedisLabsModules/redis-module-graph https://github.com/RedisLabsModules/redis-module-graph

perfmode 10y ago

You can't derive rate from latency alone. The query might be IO bound. If that's the case, you can run queries concurrently.

jnordwick 10y ago
I tend to care more about latency than throughput. FedEx has the best throughput in the world if you can wait a couple days. True though. I'm guessing with a latency number like that, the throughput is pretty bad too.
- ashwin95r 10y ago
  Exactly what https://dgraph.io https://dgraph.io is built for. Low latency, high throughput.
  - jnordwick 10y ago
    Looks interesting. That semi-declarative query language is hideous though. Reminds me of attempts to make XML into a programming language.
    - mrjn 10y ago
      (author of Dgraph) It's based on GraphQL, which is definitely not a hack like XML => language and currently catching on faster than a forest fire. Dgraph's derivation, GraphQL+- (for lack of better name) is a lot more powerful than Cypher or Gremlin. Both of the latter allow only returning lists of results; while GraphQL+- returns back an entire subgraph. Thus, all the relationships are maintained. It also allows expressing complex joins using a function like variable blocks; feeding results from one into another like you'd do in any popular language. You can try running some complex queries with GraphQL+-, it might change your view. http://play.dgraph.io/ http://play.dgraph.io/
      - jnordwick 10y ago
        Cool af Edit: I still think it looks hideous. Def try to figure it out more tomorrow.

dimfeld 10y ago

Cayley's graph data layout is most similar to a Hexastore-style [1] triple store, though IIRC it doesn't do the full six-way index that the original Hexastore paper describes. The Redis page on secondary indexing [2] has a great quick intro to what this actually entails (search the page for Hexastore). As you might guess from the Redis link, this style of graph lends itself well to KV stores, so the answer to your question #1 might be that it's a long-term decision, but the style of graph is really designed for a KV store anyway. But I haven't discussed this at all with the Cayley devs so I can't actually speak for them. I'm using it with the BoltDB backend and have been pleased with the performance overall. I haven't looked at the backends for more complex databases like Postgres in detail, but it does appear that the backend interface has potential for predicate pushdown as well. The repository's graph directory [3] contains the various backends if you want to check it out. Overall it doesn't look very difficult to add another backend type, but I haven't tried it yet. Looking at the existing SQL backend, it appears to already support MySQL, PostgreSQL, and CockroachDB (but I've tried none of these with Cayley). [1] http://www.vldb.org/pvldb/1/1453965.pdf http://www.vldb.org/pvldb/1/1453965.pdf [2] https://redis.io/topics/indexes https://redis.io/topics/indexes [3] https://github.com/cayleygraph/cayley/tree/master/graph https://github.com/cayleygraph/cayley/tree/master/graph

itamarhaber 10y ago
Speaking of which, take a look at this Redis module that marries Hexastore and neo4j-like queries: https://github.com/RedisLabsModules/redis-module-graph https://github.com/RedisLabsModules/redis-module-graph