10 ms·
Show HN: Nebula – a distributed graph database written in C++
- FillardMillmore 7y agoInteresting stuff, thanks for sharing! How did you decide to use an SQL-like query language rather than a declarative query language like Neo4j uses (Cypher query language)? What do you see as the pros and cons of that decision? Do you have any plans currently to design an ETL tool to make extracting data from a RDBMS and loading into NebulaGraph easier? I didn't see this in any of the documentation (though I did admittedly skim), but is there any sort of visual front-end built in? If not, do you have any plans to make one?
- vvern 7y agoSQL is a declarative language. Cypher is somewhat SQL-like, also declarative language which exists to make the expression of graph queries with predicates over nodes and edges easier to express.
- FillardMillmore 7y agoThank you for the correction. You are correct, SQL is a declarative language. It would've been more accurate for me to refer to Cypher as 'a more expressive declarative query language' (specifically for the graph database paradigm) than a typical SQL-like query language.
- jamie-vesoft 7y agoThanks for your interest! :) Good question re data loading! Currently there are two tools for this purpose: 1) Spark Writer which enables you to load data from DWH to Nebula Graph via Spark: https://github.com/vesoft-inc/nebula/blob/master/docs/manual-EN/3.build-develop-and-administration/3.deploy-and-administrations/server-administration/storage-service-administration/data-import/spark-writer.md https://github.com/vesoft-inc/nebula/blob/master/docs/manual... 2) CSV importer if you don't have to import very large data from RDBMS to Nebula Graph: https://github.com/vesoft-inc/nebula-importer https://github.com/vesoft-inc/nebula-importer The visual will be released by the end of this week. :)
- ragerino 7y agoApache Drill is nice for querying structured files like CSV. https://drill.apache.org/ https://drill.apache.org/
- cetra3 7y agoIt's not open source, as it is licensed under Commons Clause according to the README, which according to the FAQ is not open source. I'd be interested in knowing whether the commons clause license has been challenged as the wording is rather simple
- adrianN 7y agoI don't understand why projects don't simply use AGPL and offer a commercial license as an alternative.
- fiberoptick 7y agoThere's a subtle difference between AGPL and the Commons Clause licenses. AGPL requires network-accessible code to be disclosed & licensed under an AGPL-compatible license. The Commons Clause license outright prohibits SaaS-style offerings of the licensed code. A lot of startups licensing their code under AGPL might still have AWS et al. eat their lunch, becuase all Amazon needs to do to remain compliant is to publish any modifications made to the AGPL-ed code.
- galaxyLogic 7y agoIt seems to me the difference is not so subtle. As I understand from https://commonsclause.com/ https://commonsclause.com/ you can use Commons Clause licensed code-library as part of your commercial application without having to make your source-code available whereas with AGPL you would have to. But I may be wrong?
- orangeshark 7y agoCommons Clause is added ontop of an exisiting FOSS license. So it will be whatever requirements the base license plus an anti-commercial restriction preventing others from offering SaaS services.
- adrianN 7y agoAGPL is super-banned at all companies because lawyers deem it a huge risk. In particular it's banned at Amazon. IIRC you aren't even allowed to have AGPL software on your laptop at all.
- bane 7y agoThis is very interesting. Anybody use it? Alternatives?
- jhoechtl 7y agoAlternative:Dgraph https://dgraph.io/ https://dgraph.io/ In architecture and goals it actually closely resembles Dgraph, would love to see an (opinionated) comparison by Manish, the CEO of Dgraph
- w3clan 7y agoI had the very same feeling, dgraph is older and has a larger community plus additional features like: - geospatial features - good speed as it is based on badgerdb key value database and ristello cache library. - http library and other features One of the advantage I saw in nebula graph is security role based access which is not available in dgraph until today. I am very curious about benchmark between nebula graph and dgraph. Also what is storage system used in nebula graph.
- ayorosmage 7y agoACL is an enterprise dgraph feature: https://dgraph.io/support https://dgraph.io/support
- jhoechtl 7y agoGeospatial is on the TODO list
- jamie-vesoft 7y agoGeospatial support has already been merged to the code base. :)
- w3clan 7y agoDgraph has another disadvantage of data redundancy, where data associated with multiple index are stored multiple times for speed. Does nebula also store data multiple time for multiple index?
- rs23296008n1 7y agoThis looks good. I like the query language. Whats the status of the C++ client? https://github.com/vesoft-inc/nebula/blob/master/src/client/cpp/GraphClient.cpp https://github.com/vesoft-inc/nebula/blob/master/src/client/... I see it has connect/execute/disconnect. This seems like the minimum required. What can't I do with that basis?
- jamie-vesoft 7y agoGlad you loved the query language! The C++ client is PR ready now: https://github.com/vesoft-inc/nebula/pull/1013 https://github.com/vesoft-inc/nebula/pull/1013
- maxpert 7y agoAny production usages that can tell us about how much it can handle? How does it compare to dgraph?
- jamie-vesoft 7y agoThanks for showing interests in Nebula Graph! In some production deployments, the graph size reaches hundreds of billions of edges and data size reaches dozens of TB.
- lmeyerov 7y agoWow, not sure how I missed this! Bravo! If anyone involved would appreciate access to Graphistry to experiment with a gpu client/cloud visual analytics side (e.g., jupyter & react), let me know (leo@....com). Would love to see them together!
- jamie-vesoft 7y agoExciting! I know Graphistry from Twitter.
- slowmotarget 7y agoIt looks really neat! What's your plan on making this project viable in the long run? Do you envisage to monetize hosting, or maybe create a community vs. paid edition?
- jamie-vesoft 7y agoGlad you liked the project! Hosting service would be the main monetization method. In addition, we will be providing consulting, training and all sorts of enterprise services.
- foota 7y agoThe architecture diagram looks interesting, would love to read more about it if anyone finds something.
- rainyi2007 7y agoI found some articles about the architecture and design of the database in their repo: Overall structure: https://github.com/vesoft-inc/nebula/wiki/Nebula-Graph-Architecture-Overview https://github.com/vesoft-inc/nebula/wiki/Nebula-Graph-Archi... Storage engine design: https://github.com/vesoft-inc/nebula/blob/master/docs/manual-EN/1.overview/3.design-and-architecture/2.storage-design.md https://github.com/vesoft-inc/nebula/blob/master/docs/manual... Query engine design: https://github.com/vesoft-inc/nebula/wiki/Query-Engine-Overview https://github.com/vesoft-inc/nebula/wiki/Query-Engine-Overv... Hope that helps. :)
- ioli 7y agoThanks
- winrid 7y agoLove the query language! Very easy to dive into. Any companies using this in production?
- jamie-vesoft 7y agoGlad you loved the query language. Simplicity and versatility are our design goals for the language. Currently the project has been deployed in multiple leading internet companies in China, including Tencent, MeiTuan (Chinese Yelp), Red (Chinese Pinterest), Vivo, and so on.
- pojntfx 7y agoIt's a proprietary license, false advertising ...
- SQueeeeeL 7y agoHuh? I'm ready to dunk on any project, but that's just not true... https://github.com/vesoft-inc/nebula/blob/master/README.md#licensing https://github.com/vesoft-inc/nebula/blob/master/README.md#l...
- jamie-vesoft 7y agoThanks for your feedback! While our original intention is to provide a real open-source graph database project for the community, we also want to prevent cloud vendors from monetizing the project without contributing back to the community. Exactly like what's explained in this TechCrunch article: https://techcrunch.com/2018/09/07/commons-clause-stops-open-source-abuse/ https://techcrunch.com/2018/09/07/commons-clause-stops-open-... That being said, Common Clause seems to be the only license that can be used. Quote the article: "Academics, hobbyists or developers wishing to use a popular open-source project to power a component of their application can still do so. " However, we will seriously consider the license issue. Please do let us know if you know any better licenses that can be used. Much appreciated!
- dang 7y agoWe changed the URL from https://github.com/vesoft-inc/nebula https://github.com/vesoft-inc/nebula to the project page.
- jamie-vesoft 7y agoHey, thanks for letting me know. May I ask why?
- dang 7y agoBecause it's more project-specific and thus stands out more. There are a ton of Github links posted to HN, obviously. If you'd rather we switch to the other URL, let us know and we can do that.
- jamie-vesoft 7y agoAh I see. Thanks for the explanation! Could you please change the URL back to the GitHub URL? Also the title? Much appreciated!
- dang 7y agoI've changed the URL back from https://nebula-graph.io/ https://nebula-graph.io/. The submitted title ("Show HN: An open-source distributed graph database written in C++") led to lots of arguments, so we changed it in accordance with HN's rules about baity titles. That's in https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html.
- jamie-vesoft 7y agoThanks for the explanation. I thought that was the reason you changed the title. Makes sense to me. :)
- essive 7y agoWhy is this project tied so specifically to China? Honest question here....
- flohofwoe 7y agoWhat are those specific ties other than the company behind the project being located in China?
- apta 7y agoStill a red flag.
- SQueeeeeL 7y agoIs it? That's like saying that Intel being located in the US is a red flag for them leaving backdoors in their hardware... oohhh https://en.wikipedia.org/wiki/RDRAND#Reception https://en.wikipedia.org/wiki/RDRAND#Reception
- apta 7y agoAnd China has backdoors in their stuff. We shouldn't be using it.
- deleted 7y ago[deleted]
- sweetdreamerit 7y ago"Five-starred Red Flag", indeed ;) https://en.wikipedia.org/wiki/Flag_of_China https://en.wikipedia.org/wiki/Flag_of_China
- flohofwoe 7y agoSo basically "only buy American"? ;) I don't want to defend that company or the product, or the country they operate from, but the source code is all on github under a permissive license and thus can easily be auditioned for government backdoors. Where's the problem?
- gibsonf1 7y agoSPARQL?
- ablekh 7y agoLooks interesting. Good luck! Couple of questions: 1) why did you decide to create your own graph query language instead of trying to follow recent graph query languages standardization trend (e.g., see https://www.tigergraph.com/2019/02/25/the-road-to-standardized-graph-query-language-gql-part-1 https://www.tigergraph.com/2019/02/25/the-road-to-standardiz... and https://www.tigergraph.com/2019/03/15/the-road-to-a-standardized-graph-query-language-gql-part-2); https://www.tigergraph.com/2019/03/15/the-road-to-a-standard... 2) do you plan to support semantic features [i.e., RDF/S, Gremlin, SPARQL, inference]; 3) do you plan to have SDKs for popular languages [e.g., Python, TypeScript]; 4) did you benchmark (or plan to do so) Nebula Graph against competition in terms of performance [i.e., data import, query throughput and latency, inference speed, if/when supported]; 5) have you figured out which features will remain open source and which ones will be enterprise-only (I assume that you plan to follow the Open Core business model)? Thanks!
- jamie-vesoft 7y agoThanks for the questions! Good ones indeed. 1)nGQL will be compatible with the international standard GQL. 2) Currently no 3) So far the following SDKs are available: Python https://github.com/vesoft-inc/nebula-python Java https://github.com/vesoft-inc/nebula-java Go https://github.com/vesoft-inc/nebula-go 4) We are working on the benchmark. Stay tuned. 5) Yes we'll follow the Open Core business model like you said. All the features that are open source today will remain open source. :) Please let me know if you have any further questions.
- ablekh 7y agoIt's my pleasure! Thank you for the clarifications. I will definitely follow your efforts. :-)
- jamie-vesoft 7y agoMuch appreciated! You may find us on Twitter to follow the most recent updates.:) Also, welcome to be a contributor!
- krak12 7y agoAnyone here questioning the license, just stole it, its a chinese copycat from another company, so feel free to ignore developer, watch backdoors and never use it in production
- shermanye 7y agoNice to meet everyone here. As a newcomer, I would like to introduce ourselves a little bit. Nebula is inspired by the Facebook internal project Dragon (https://engineering.fb.com/data-infrastructure/dragon-a-distributed-graph-query-engine/ https://engineering.fb.com/data-infrastructure/dragon-a-dist...). Fortunately I was one of the founding members of the project. The project was started in 2012. Since then I've been spent all my time working on the graph databases. The goal of Nebula is to be a general-purposed, distributed graph database. We welcome any positive feedback and technical discussion. We would love to learn to the community and to provide a product which truly satisfies customers' needs.
- ragerino 7y agoI gave the tutorial using docker a try. The SQL like query language is OK until it gets to doing queries with where clauses on the data. Some form of help and auto completion would be great. I recommend to align with the GQL standard: https://www.gqlstandards.org/ https://www.gqlstandards.org/ And Apache Tinkerpop. E.g. a RDF Sail implementation for Nebula. http://tinkerpop.apache.org/ http://tinkerpop.apache.org/
- jamie-vesoft 7y agoThanks so much for trying Nebula! We really appreciate it. Sorry about the where clause issue. Do you mind bringing an issue in this regard on our GitHub repo? So that we can assign it to relevant staff. As to the query language, thanks for your suggestion and nGQL will surely be aligned with the GQL standard. We are keeping a close eye on it. :) We are planning to support OpenCypher in the first half of 2020 and TinkerPop would be the next. Thanks again! Here's our slack group btw and you may raise any question there: https://join.slack.com/t/nebulagraph/shared_invite/enQtNjIzMjQ5MzE2OTQ2LTM0MjY0MWFlODg3ZTNjMjg3YWU5ZGY2NDM5MDhmOGU2OWI5ZWZjZDUwNTExMGIxZTk2ZmQxY2Q2MzM1OWJhMmY# https://join.slack.com/t/nebulagraph/shared_invite/enQtNjIzM...