3 ms·

We have Elasticsearch as a generic document search engine, each document has a non-trivial amount of properties (let's say 50 or so). It's incredibly performant

We have Elasticsearch as a generic document search engine, each document has a non-trivial amount of properties (let's say 50 or so). It's incredibly performant for all sort of searches, the details of each of which this solution wasn't specifically designed for, hence me calling it a generic search engine. Every time I contemplate of bringing graph relationships, that exist between these documents, into the mix, I get stuck. Elasticsearch doesn't quite do graph (natively), but the graph databases I tried don't do properties too well (OrientDB, Neo4J). I'm not talking about one or two properties, but multiple properties across multiple hops in the graph that I envision querying for. Let alone full text searches. I emailed back and forth with the helpful folks at Orient, but it always came down to optimising for specific queries, gone the "generic". Is anyone solving that problem? Cayley?

reactor 10y ago
Worth taking a look at https://arangodb.com https://arangodb.com or http://janusgraph.org/ http://janusgraph.org/ (Fork of Titan graph db)
solipsism 10y ago
I guess you've considered doing the graph search and the text search separately and joining the results?
- sverhagen 10y ago
  Yes, but I can easily reason myself into scenarios where the intermediate result would be prohibitively large.
mrjn 10y ago
(author of Dgraph) Dgraph's retrieval is pretty fast, so looking up properties is trivial. It also supports indexing various data types: supports full-text search, term matching and regexps on strings, inequality and sorting on ints, floats, dates etc. https://docs.dgraph.io/v0.7.4/query-language/#functions https://docs.dgraph.io/v0.7.4/query-language/#functions One of our users is on the path to switch to Dgraph from Elastic Search. So, I'd say try Dgraph out and see if that'd help your use-case. I think it should. And if Dgraph is missing something that you need from ES, feel free to file an issue. Happy to address it.
nickstefan12 10y ago
If you'd be okay with a directed acyclic graph, then SQL can work. Basically "modeling trees in SQL". Specifically, you could use postgresql for the edge traversing and its jsonb column to store searchable attributes.
janemanos 10y ago
Sounds like something ArangoDB could be a good solution for. Full disclosure I'm from ArangoDB team and happy to help. If you like just drop me a line to jan.stuecke (at) arangodb.com
- lolive 10y ago
  Being a graph database user, i always have to manage a replication of the "molecules"of my graph in ES for a user-friendly search experience. Ca arangodb help for such a use case? Or may be dgraph?
  - mrjn 10y ago
    Dgraph does automatic replication, for providing fault tolerance. It's baked in pretty deep into Dgraph. We use consensus algorithm, so all your writes are atomically consistent (not eventually consistent). https://docs.dgraph.io/v0.7.4/deploy/ https://docs.dgraph.io/v0.7.4/deploy/
  - janemanos 10y ago
    With ArangoDB you can choose between synchronous replication and asynchronous. With the Agency of ArangoDB you also have a RAFT based consensus protocol which holds the state of the cluster. My team mate wrote a nice article about our approach. You might want to have a look: https://www.arangodb.com/2017/01/reaching-harnessing-consensus-arangodb/ https://www.arangodb.com/2017/01/reaching-harnessing-consens.... In single instance you have full transactional guarantees with multi collection and multi document transactions. In cluster mode we provide single document transactions. More guarantees will follow.

reactor 10y ago

Worth taking a look at https://arangodb.com https://arangodb.com or http://janusgraph.org/ http://janusgraph.org/ (Fork of Titan graph db)

solipsism 10y ago

I guess you've considered doing the graph search and the text search separately and joining the results?

sverhagen 10y ago
Yes, but I can easily reason myself into scenarios where the intermediate result would be prohibitively large.

mrjn 10y ago

(author of Dgraph) Dgraph's retrieval is pretty fast, so looking up properties is trivial. It also supports indexing various data types: supports full-text search, term matching and regexps on strings, inequality and sorting on ints, floats, dates etc. https://docs.dgraph.io/v0.7.4/query-language/#functions https://docs.dgraph.io/v0.7.4/query-language/#functions One of our users is on the path to switch to Dgraph from Elastic Search. So, I'd say try Dgraph out and see if that'd help your use-case. I think it should. And if Dgraph is missing something that you need from ES, feel free to file an issue. Happy to address it.

nickstefan12 10y ago

If you'd be okay with a directed acyclic graph, then SQL can work. Basically "modeling trees in SQL". Specifically, you could use postgresql for the edge traversing and its jsonb column to store searchable attributes.

janemanos 10y ago

Sounds like something ArangoDB could be a good solution for. Full disclosure I'm from ArangoDB team and happy to help. If you like just drop me a line to jan.stuecke (at) arangodb.com

lolive 10y ago
Being a graph database user, i always have to manage a replication of the "molecules"of my graph in ES for a user-friendly search experience. Ca arangodb help for such a use case? Or may be dgraph?
- mrjn 10y ago
  Dgraph does automatic replication, for providing fault tolerance. It's baked in pretty deep into Dgraph. We use consensus algorithm, so all your writes are atomically consistent (not eventually consistent). https://docs.dgraph.io/v0.7.4/deploy/ https://docs.dgraph.io/v0.7.4/deploy/
- janemanos 10y ago
  With ArangoDB you can choose between synchronous replication and asynchronous. With the Agency of ArangoDB you also have a RAFT based consensus protocol which holds the state of the cluster. My team mate wrote a nice article about our approach. You might want to have a look: https://www.arangodb.com/2017/01/reaching-harnessing-consensus-arangodb/ https://www.arangodb.com/2017/01/reaching-harnessing-consens.... In single instance you have full transactional guarantees with multi collection and multi document transactions. In cluster mode we provide single document transactions. More guarantees will follow.