6 ms·
This is very interesting. Anybody use it? Alternatives?
by bane 7y ago
This is very interesting. Anybody use it? Alternatives?
- jhoechtl 7y agoAlternative:Dgraph https://dgraph.io/ https://dgraph.io/ In architecture and goals it actually closely resembles Dgraph, would love to see an (opinionated) comparison by Manish, the CEO of Dgraph
- w3clan 7y agoI had the very same feeling, dgraph is older and has a larger community plus additional features like: - geospatial features - good speed as it is based on badgerdb key value database and ristello cache library. - http library and other features One of the advantage I saw in nebula graph is security role based access which is not available in dgraph until today. I am very curious about benchmark between nebula graph and dgraph. Also what is storage system used in nebula graph.
- ayorosmage 7y agoACL is an enterprise dgraph feature: https://dgraph.io/support https://dgraph.io/support
- jhoechtl 7y agoGeospatial is on the TODO list
- jamie-vesoft 7y agoGeospatial support has already been merged to the code base. :)
- w3clan 7y agoDgraph has another disadvantage of data redundancy, where data associated with multiple index are stored multiple times for speed. Does nebula also store data multiple time for multiple index?
- campoy 7y agoHi there, out of curiosity what do you mean the data is stored multiple times for speed? We (I work at Dgraph) have data redundancy when you have multiple replicas for a given group - but that's an optional feature. Thanks!
- jamie-vesoft 7y agoThanks for asking! Sorry I missed this question earlier. Nebula doesn't store data multiple times for index. And here's how the indexing works in Nebula Graph: You are allowed to create multiple indexes for one tag or edge type in Nebula Graph. For example, if a vertex has 5 properties attached to it, then you can create one index for each if it's necessary for you. Both indexes and the raw data are stored in the same partition with their own data structure for quick query statement scanning. Whenever there are "where" clause/syntax in the queries, the index optimizer decides which index file should be traversed.
- jamie-vesoft 7y agogeospatial is also available in Nebula Graph.:) As to the storage system, Nebula Graph is based on multi-group raft and RocksDB.
- w3clan 7y agoCan you add support for module based storage system like, if someone wants to use badgerdb or leveldb or any other storage system instead of rocksdb
- jamie-vesoft 7y agoYes, Nebula Graph supports multiple backend storages by design. So theoretically you are able to use whatever storage you want for whichever graph space in Nebula Graph. You may take a look at this article about the design of our storage engine: https://github.com/vesoft-inc/nebula/blob/master/docs/manual-EN/1.overview/3.design-and-architecture/2.storage-design.md https://github.com/vesoft-inc/nebula/blob/master/docs/manual... In 2020 we will be working on more plugins. You may stay tuned if that interests you. :)
- mrjn 7y ago(Manish here) Don't know much about Nebula. Feels quite inspired by Dgraph.
- shermanye 7y ago(Sherman here. I'm the founder of Nebula) Nice to meet you here, Manish. Nebula is actually inspired by the Facebook internal project Dragon (https://engineering.fb.com/data-infrastructure/dragon-a-distributed-graph-query-engine/ https://engineering.fb.com/data-infrastructure/dragon-a-dist...). Fortunately I was one of the founding members of the project. The project was started in 2012. We never heard of dgraph at that time. So I'm not sure who was inspired :-) The goal of Nebula is to be a general graph database, not just a knowledge graph database. There are some fundamental differences between the two. We welcome any positive feedback and technical discussion. We would love to learn to the community and to provide a product which truly satisfies customers' needs.
- jhoechtl 7y ago> The goal of Nebula is to be a general graph database, not just a knowledge graph database. There are some fundamental differences between the two. I am by no means a Graph expert but what are some of the mentioned fundamental differences?
- emmanueloga_ 7y agoCheck the list of implementations of SPARQL [1]. One of the most interesting picks: RDF4j (java based). It can connect to a lot of different SPARQL servers, but the rdf4j Native Store should be good enough for data sets in the order of the "100 million triples", according to the docs. I don't know much about it, but not long ago they announced integrated support for "federated queries", which means that if you data set can't fit in a single node, they have a solution to query different servers in the same query [2]. I'm slowly learning through the forest of related technologies, one of the most useful is SHACL [3], which is a language to validate and extract pieces of the graph that match a pattern (very loosely, think a "schema" for graphs). 1: https://en.wikipedia.org/wiki/List_of_SPARQL_implementations https://en.wikipedia.org/wiki/List_of_SPARQL_implementations 2: https://rdf4j.org/news/2019/10/15/fedx-joins-rdf4j/ https://rdf4j.org/news/2019/10/15/fedx-joins-rdf4j/ 3: https://rdf4j.org/documentation/programming/shacl/ https://rdf4j.org/documentation/programming/shacl/
- rapnie 7y agoBefore using RDF for graphs one should inform themselves on the differences between labeled property graphs and triple stores, and choose the model that best fit their use case. Take your time and beware of objectivity of article. Vendors try to lure you in. Following has some good info (but a Neo4j bias): https://dzone.com/articles/rdf-triple-stores-vs-labeled-property-graphs-whats https://dzone.com/articles/rdf-triple-stores-vs-labeled-prop...
- emmanueloga_ 7y agoGood point. Funny you mention that article: I remember encountering both that article and another one that provides some counterpoints! [1] Also, both those articles are a bit old: RDF* ([2],[3]) is a new extension for RDF that makes it easier to accomplish the same kind of things you can do with property graphs. RDF4j has support for RDF* in the roadmap! [4]. To me, the fact that RDF is 1) a simpler and more general model and 2) an open standard with multiple free and commercial implementations; makes RDF a more a attractive option than locking into a single proprietary implementation like Neo4j. -- 1: http://www.snee.com/bobdc.blog/2018/04/reification-is-a-red-herring.html http://www.snee.com/bobdc.blog/2018/04/reification-is-a-red-... 2: http://olafhartig.de/slides/RDFStarInvitedTalkWSP2018.pdf http://olafhartig.de/slides/RDFStarInvitedTalkWSP2018.pdf 3: http://blog.liu.se/olafhartig/2019/01/10/position-statement-rdf-star-and-sparql-star/ http://blog.liu.se/olafhartig/2019/01/10/position-statement-... 4: https://github.com/eclipse/rdf4j/issues/1484 https://github.com/eclipse/rdf4j/issues/1484
- sfgweilr4f 7y agoI'd say dgraph (https://dgraph.io/ https://dgraph.io/) is closest competitor but Neo4j (https://neo4j.com https://neo4j.com) as well which has longer heritage. I'd also include redis because of the graph module (https://oss.redislabs.com/redisgraph/ https://oss.redislabs.com/redisgraph/). I've likely missed a bunch of others. Add them as I'm interested in graph db and have only scratched the surface myself.
- rapnie 7y agoThere is also decentralized Gun DB: https://gun.eco/ https://gun.eco/ OrientDB is an alternative to Neo: https://orientdb.org/ https://orientdb.org/ There are a bunch of DB's compatible with Tinkerpop and e.g. query-able with Gremlin: http://tinkerpop.apache.org/ http://tinkerpop.apache.org/
- rapnie 7y agoThere is also Weaviate, still in development, which has a flavor of GraphQL for querying: https://github.com/semi-technologies/weaviate https://github.com/semi-technologies/weaviate And this awesome page has some good entries: https://github.com/jbmusso/awesome-graph/blob/master/README.md https://github.com/jbmusso/awesome-graph/blob/master/README....
- lmeyerov 7y agoWe help bring gpu visual analytics & investigation automation to users of all sorts of graph DBs (think tableau & servicenow for graph), so based on our enterprise/big tech/gov/startup interactions: 1. Shortlist (and in no order): Neo4j, AWS Neptune, Datastax Graph, TigerGraph, Azure CosmosDB, and JanusGraph (Titan fork) are the ones we see the most in practice, and not in production but rumor-mill, Dgraph, RedisGraph, & ArangoDB. The three-and-four-letter types seem to roll their own, for better or worse. There are also some super cool ones that don't get visibility outside of the HPC+DoD world, like Stinger & Gunrock. Interestingly, the reality is a ton of our graph users aren't even on graph DBs (think Splunk/ELK/SQL), and for data scientists, just do ephemeral Pandas/Spark. As someone from the early days of the end-to-end GPU computing movement, we're incorporating cuGraph (part of nvidia rapids.ai) into our middle tier, so you get to transparently benefit from it while looking at data in any of the above. 2. I now slice graph DB's more in terms of OLTP (neo4j, janus, neptune, maybe tiger) vs OLAP (spark graphx, cugraph) vs batch (janus, tiger) vs friendly BI/data science (neo4j) vs friendly app dev / multi-modal add-on (CosmosDB, Neo4j, Arango, Redis). Curious to see how this goes -- given the number of contributors, I'm guessing it's doing well in at least one of these. +1 to hearing reports from others!
- bane 7y agoThanks, I really appreciate the comprehensive write up of what your team is seeing. Any chance of a longer blog post that expands on this, especially pro-cons and performance?
- lmeyerov 7y agoYes, that is a great idea!
- derefr 7y agoFor someone who just wants to run some (intensive) OLAP graph queries on the “graph formulation” of a relational or hierarchical dataset every once in a while (maybe batch, maybe user-initiated, but either way <1QPS), but doesn’t yet have a graph DB and doesn’t really want to maintain their data in a canonical graph formulation, which type of graph DB would you recommend as the simplest-to-maintain, simplest-to-scale “adjunct” to their existing infra? I.e. what’s the graph DB that best fits the use-case equivalent to “having your data in an RDBMS and then running an indexer agent to feed ElasticSearch for searching”?
- jamie-vesoft 7y agoCurrently the project has been deployed in multiple leading internet companies in China, including Tencent, MeiTuan (Chinese Yelp), Red (Chinese Pinterest), Vivo, and so on.
- bane 7y agoThat's pretty impressive. I'd love to see some details blog posts about setting it up, or using it in production (things to watch out for, good practices for provisioning hardware, etc.).
- jamie-vesoft 7y agoThere is a Getting Started series in the GitHub wiki page: https://github.com/vesoft-inc/nebula/blob/master/docs/manual-EN/1.overview/2.quick-start/1.get-started.md https://github.com/vesoft-inc/nebula/blob/master/docs/manual... Check out the Getting started YouTube video here if you prefer video tutorials: https://www.youtube.com/channel/UC73V8q795eSEMxDX4Pvdwmw https://www.youtube.com/channel/UC73V8q795eSEMxDX4Pvdwmw Also some FAQs: https://github.com/vesoft-inc/nebula/blob/master/docs/manual-EN/1.overview/2.quick-start/2.FAQ.md https://github.com/vesoft-inc/nebula/blob/master/docs/manual... If you are interested in the architectural design of the project, here are some articles for your reference: Overview: https://github.com/vesoft-inc/nebula/blob/master/docs/manual-EN/1.overview/3.design-and-architecture/1.design-and-architecture.md https://github.com/vesoft-inc/nebula/blob/master/docs/manual... Storage engine: https://github.com/vesoft-inc/nebula/blob/master/docs/manual-EN/1.overview/3.design-and-architecture/2.storage-design.md https://github.com/vesoft-inc/nebula/blob/master/docs/manual... Query engine: https://github.com/vesoft-inc/nebula/blob/master/docs/manual-EN/1.overview/3.design-and-architecture/3.query-engine.md https://github.com/vesoft-inc/nebula/blob/master/docs/manual... Feel free to contact us if anything is missing. :)
- gibsonf1 7y agoWe're using Blazegraph (The db that Amazon Neptune uses) with great success. We only use the Sparql api with quads so we get nested graphs.