6 ms·

Show HN: Memgraph – Transactional, in-memory, cypher-compatible graph database

89 points by karimtr 6y ago

mbuda 6y ago
Hey everyone, Memgraph CTO here. This is a massive milestone for us that took almost 3 years to reach and I would like to thank everyone who helped us get here! If you have any questions or feedback, I’ll be around all day to answer :D
- JPKab 6y ago
  Congrats. I'll be kicking the tires with this for some projects at my company.
Martinsos 6y ago
Congrats :)! How does this compare to Neo4J and what would be some typical use cases?
- mbuda 6y ago
  Thank you! So, Memgraph is compatible with Cypher (http://www.opencypher.org http://www.opencypher.org) query language and Bolt protocol (https://boltprotocol.org https://boltprotocol.org) which means that existing tools and workloads will work. When it comes to the differences, because of our in-memory first approach, low-level implementation and optimizations Memgraph is more suitable for real-time analytical graph workloads. Please let me know if this makes sense or you would like to learn more about specific details. Out of curiosity, do you have an interesting real-time graph use-case?
  - mtt87 6y ago
    This looks awesome! How much Cypher does Memgraph support? Also, what are the difference compared with Neo4j's Cypher?
    - mbuda 6y ago
      Memgraph supports more than 80% of openCypher TCK scenarios (https://github.com/opencypher/openCypher/tree/master/tck https://github.com/opencypher/openCypher/tree/master/tck). For more details about the actual differences, please take a look here https://docs.memgraph.com/memgraph/reference-overview/differences https://docs.memgraph.com/memgraph/reference-overview/differ.... The biggest difference is that Memgraph supports loading custom query modules implemented in almost any programming language and compiled into a dynamic library (.so file). Or, running query modules written as Python scripts.
phowson83 6y ago
Can I use all NetworkX features and library functions from within memgraph, e.g. can I apply k_components to graph held within memgraph?
- mbuda 6y ago
  Yes, Memgraph has an embedded Python interpreter so you can run any NetworkX graph algorithm or any other Python code. The code has access to the whole graph, so a bunch of options exists. The entire Python integration is still in the early days, and we have a bunch of plans on how to improve further. What Python libraries do you generally use?
M_jolnir 6y ago
Would Memgraph be a good app to geolocate all of the data involved with a graph database? Example: geolocate all of the electric distribution networks around the country7globe that interconnect with the values of the graph?
- karimtr 6y ago
  Karim from Memgraph here. Thanks for the questions. Are you referring to geospatial capabilities? If I understand correctly, you would like to add geospatial coordinates to specific elements in the graph (e.g. power stations, power lines etc) and be able to search and query for specific areas (e.g. countries, cities, etc)? If that's the case, although we don't have direct support for Geospatial data format, you can easily encode it into your graph by setting coordinate as a property on your nodes and edges. Does this answer your question?
  - M_jolnir 6y ago
    Yes Karim, exactly that! :) Thank you for the answer
enzotar 6y ago
Can Memgraph be used as an embedded database similar to SQLite?
- mbuda 6y ago
  The intended usage isn't that one because Memgraph is built as a server database system. The communication has to go via the binary protocol. But, because Memgraph is implemented in C/C++ and there is the C API for the query modules already in-place, offering an embedded solution is possible. This feature isn't high on the priorities list at the moment. What would be your target platforms? Mobile or maybe something on the IoT side?
  - enzotar 6y ago
    Mobile
  - acarrera94 6y ago
    +1 for mobile
enzotar 6y ago
Will there be an open-source edition?
- karimtr 6y ago
  Karim from Memgraph here. That's a good question we get really often. As you can imagine we had our hands full for the past few years building a DB from scratch, and so we didn't really have the resources to put together an open-source project the way we imagine it. It's something we'll explore this year to try to come up with the best way to do it.
kamranjon 6y ago
How does this compare to RedisGraph?
- karimtr 6y ago
  Good question :D We haven't spent much time trying out RedisGraph, nor we ran any kind of benchmarks at his point, but from what I can see, I would say that the main differences would: 1. Memgraph is a native GraphDB which means that it's specifically engineered to support graph and graph only whereas RedisGraph is a module that runs on top of key-value store (nothing wrong with that, but it might have some limitation when it comes to performance on complex graph algorithms and traversals) 2. Cypher coverage – We cover about 80% of the cypher query language, where I think RedisGraph covers only a small portion (But I hear that they're improving on that) Apart from that, I guess you would really need to test both when for specific use-cases to judge performance and scalability. We'll try to provide some benchmarks in the next few weeks. I hope this helps :D Have you used RedisGraph? Any thoughts?
mralj 6y ago
This looks really cool, congrats on publishing DB :D
- mbuda 6y ago
  Thank you!
wiradikusuma 6y ago
How does this compare to Dgraph? Most recent discussion about it: https://news.ycombinator.com/item?id=23031762 https://news.ycombinator.com/item?id=23031762 Also, does it support GraphQL?
- deleted 6y ago
  [deleted]
- karimtr 6y ago
  To answer your first question, the main difference between Memgraph and Dgraph seem to be: 1. The graph model: Property graph for Memgraph vs RDF for DGraph (Depending on the use-case you have one model might be better than the other. E.g ontologies are better served with RDF, whereas graphs with a lot of properties and labels are better served by the property graph model) 2. Memgraph is an in-memory first system where is Dgraph is a disk-based system. Again depending on your use-case and the performance you are looking for, an in-memory system might be better suited. 3. Query language & Ecosystem: we support Cypher and the Bolt Protocol (Same as Neo4j) so we work with a lot of the existing graph tools. In terms of performance, we don't have official benchmarks yet but we have a few clients that test Memgraph and Dgraph and reported a 3-5x in read performance and about an 8x in write performance for their specific workloads. I hope this answers your question.
  - karimtr 6y ago
    Sorry, I missed the second part of you question. You should be able to use the Neo4j GraphQL with Memgraph although we didn't fully test it yet(https://github.com/neo4j-graphql/neo4j-graphql https://github.com/neo4j-graphql/neo4j-graphql).
  - micheldiz 6y ago
    Hi guys, I'm community support at Dgraph. Reading that question. I come to clarify some points. First of all, I don't particularly know Memgraph internally. So, I'll stick to Dgraph and related only. About Karimtr's reply 1. The graph model: Although Dgraph uses RDF as input data. Dgraph is not a Triple Store per se - and the RDF we have is a customized version, which means that it is not 100% compatible with any RDF model (e.g. Turtle RDF) - But eventually, many RDF syntaxes may be compatible. The decision to use RDF was made a long time ago, for reasons that I am particularly unaware of. It was long before I joined Dgraph. We also accept JSON as data input. By the way, I also don't understand why Neo4j uses CSV as input data since it is not a Graph standard. RDF itself would be more acceptable than CSV. I assume they use CSV for strategic reasons. 1.1 In practice Dgraph is technically a "Property graph" like. There are no fundamental differences between the Dgraph's graph model with Neo4j other than the language itself and the way the data is stored and injected. 1.2 In Dgraph the data is stored in KV using BadgerDB. 1.3 Ontologies can be represented in any GraphDB. The difference is that Triple Store DBs have a language created to infer data specifically with the concept of ontology. And Triple Stores has standardized data input for this. 2. That's right. However, I think we will soon have the option to keep it in memory. But I don't particularly know how useful this is. Today you can keep some Memory first data, but with the guarantee that they will be saved on disks. This helps in performance when there is no use of NVMe. 3. We have GraphQL+- which is a rich language and inspired by GraphQL. And we also have GraphQL which is an "API" language that is now native in Dgraph. A friendly front-end language. And it works "out of the box" once you mount your schema. Dgraph creates a CRUD model based on your Schema. This reduces production time for your application and less logic on your business side. We are still adding "Black Magic" so that the experience in producing APPs is exceptional. And less code typing. About performance. I suggest doing a test against ludicrous mode https://discuss.dgraph.io/t/sharing-some-numbers-from-the-ludicrous-mode/6310?u=micheldiz https://discuss.dgraph.io/t/sharing-some-numbers-from-the-lu... Cheers.
VRomanov89 6y ago
How do you handle high-availability? Also, do you have some sort of persistency model for the community version which doesn't have HA?
- dtomicevic 6y ago
  Dominik from Memgraph here. Good questions! All editions of Memgraph persist data to the disk via write-ahead logging (WAL) and periodic snapshots for log compaction so even though Memgraph is designed to be in-memory first, data is always backed up to disk. If you enable asynchronous (periodic fsync) WAL, you can trade off a small window of durability for better performance. Before 1.0, Memgraph leveraged the RAFT consensus algorithm for HA which worked great but had some performance implications. Based on feedback from our users and customers, we have decided to switch our HA implementation to the streaming replication model (similar to PostgreSQL) with automatic failover.
  - greenrobot 6y ago
    Interesting. Would you mind to go into details about dropping RAFT? It would make a great tech blog post imho, but maybe you could share some high level insights on that? Related to that: have you considered extensions to the RAFT standard e.g. pipelining?
bavell 6y ago
Congrats on reaching 1.0! I'm always interested in new GraphDB solutions and glad to see the Cypher integration and Neo4j wire compatibility here. I'm excited seeing the progress being made in this space recently (this, redisgraph, etc) - the wider ecosystem is still immature but promising.
- dtomicevic 6y ago
  Thank you! We are here to do our fair share and provide the community with the right tools to solve different kinds of graph problems. Exciting space nevertheless and growing fast!
bkubicka 6y ago
Congrats on the launch! Which graph algorithms do you support?
- mbuda 6y ago
  Thank you! Memgraph has built-in BFS, DFS, and weighted-shortest path. These can leverage the query planning (use the details about data distribution to perform better). Since query modules are introduced, algorithms like Page Rank could be easily implemented. Memgraph offers a couple of them out of the box. Last but not least, all algorithms from the Python eco-system could be run inside Memgraph.
abuteau 6y ago
Congrats on the launch! Do you have any plans to release a NodeJs Driver?
- mbuda 6y ago
  Thank you! So far, Memgraph offers and supports C/C++ library (https://github.com/memgraph/mgclient https://github.com/memgraph/mgclient) and Python binding (https://github.com/memgraph/pymgclient https://github.com/memgraph/pymgclient). Node.js is going to come shortly. Please stay tuned!
bionhoward 6y ago
That cloud pricing page is totally useless because it provides zero detail on what you get for your money, how much you need for a given use case (M8? M32?), and forces you to calculate monthly cost yourself. Further, without any autoscaling/Serverless option, we wind up paying for, polluting for, and wasting energy for 24/7 instances at fixed capacity. Cool product, but the deal is unclear and inflexible, so DynamoDB or Postgres are still better options When is someone going to provide an actual serverless graph db? It’s unbelievable how much masturbatory self-congratulation goes on in the graph database community while Serverless Postgres and Dynamo are exponentially cheaper and better. Is Cypher really worth a 100x-1000x price hike?
FShieldheart 6y ago
Congratulations on your release! Keep up the good work!
- dtomicevic 6y ago
  Thank you!
JPKab 6y ago
"An integrated ecosystem that would allow data scientists to easily leverage existing data science and machine learning tools to build graph-powered applications with minimum friction." Graph databases have as much to do with data science as they do with any software applications. Going to guess a VC made them put that there. Definitely excited to try it out for a project I'm working on with DAG optimizations.
RocketSyntax 6y ago
This is huge! I'd like to see the roadmap for graph algorithms and graph neural nets! I can provide feedback on both.
- mbuda 6y ago
  Yea, we're still discussing the exact roadmap on this side. In the short term, the plan is to integrate better with the whole ecosystem. We are still exploring options on the graph NN side. Do you have any specific algorithms to point out? :D
zvone 6y ago
Hey everyone, Two questions, please. What about C#? Do you have some API? What about spatial search? Do you have that integrated in API? Thanks
bfrit 6y ago
This is awesome!

89 points by karimtr 6y ago

Hey everyone, Memgraph CTO here. This is a massive milestone for us that took almost 3 years to reach and I would like to thank everyone who helped us get here! If you have any questions or feedback, I’ll be around all day to answer :D

JPKab 6y ago
Congrats. I'll be kicking the tires with this for some projects at my company.

Martinsos 6y ago

Congrats :)! How does this compare to Neo4J and what would be some typical use cases?

mbuda 6y ago
Thank you! So, Memgraph is compatible with Cypher (http://www.opencypher.org http://www.opencypher.org) query language and Bolt protocol (https://boltprotocol.org https://boltprotocol.org) which means that existing tools and workloads will work. When it comes to the differences, because of our in-memory first approach, low-level implementation and optimizations Memgraph is more suitable for real-time analytical graph workloads. Please let me know if this makes sense or you would like to learn more about specific details. Out of curiosity, do you have an interesting real-time graph use-case?
- mtt87 6y ago
  This looks awesome! How much Cypher does Memgraph support? Also, what are the difference compared with Neo4j's Cypher?
  - mbuda 6y ago
    Memgraph supports more than 80% of openCypher TCK scenarios (https://github.com/opencypher/openCypher/tree/master/tck https://github.com/opencypher/openCypher/tree/master/tck). For more details about the actual differences, please take a look here https://docs.memgraph.com/memgraph/reference-overview/differences https://docs.memgraph.com/memgraph/reference-overview/differ.... The biggest difference is that Memgraph supports loading custom query modules implemented in almost any programming language and compiled into a dynamic library (.so file). Or, running query modules written as Python scripts.

phowson83 6y ago

Can I use all NetworkX features and library functions from within memgraph, e.g. can I apply k_components to graph held within memgraph?

mbuda 6y ago
Yes, Memgraph has an embedded Python interpreter so you can run any NetworkX graph algorithm or any other Python code. The code has access to the whole graph, so a bunch of options exists. The entire Python integration is still in the early days, and we have a bunch of plans on how to improve further. What Python libraries do you generally use?

M_jolnir 6y ago

Would Memgraph be a good app to geolocate all of the data involved with a graph database? Example: geolocate all of the electric distribution networks around the country7globe that interconnect with the values of the graph?

karimtr 6y ago
Karim from Memgraph here. Thanks for the questions. Are you referring to geospatial capabilities? If I understand correctly, you would like to add geospatial coordinates to specific elements in the graph (e.g. power stations, power lines etc) and be able to search and query for specific areas (e.g. countries, cities, etc)? If that's the case, although we don't have direct support for Geospatial data format, you can easily encode it into your graph by setting coordinate as a property on your nodes and edges. Does this answer your question?
- M_jolnir 6y ago
  Yes Karim, exactly that! :) Thank you for the answer

enzotar 6y ago

Can Memgraph be used as an embedded database similar to SQLite?

mbuda 6y ago
The intended usage isn't that one because Memgraph is built as a server database system. The communication has to go via the binary protocol. But, because Memgraph is implemented in C/C++ and there is the C API for the query modules already in-place, offering an embedded solution is possible. This feature isn't high on the priorities list at the moment. What would be your target platforms? Mobile or maybe something on the IoT side?
- enzotar 6y ago
  Mobile
- acarrera94 6y ago
  +1 for mobile

enzotar 6y ago

Will there be an open-source edition?

karimtr 6y ago
Karim from Memgraph here. That's a good question we get really often. As you can imagine we had our hands full for the past few years building a DB from scratch, and so we didn't really have the resources to put together an open-source project the way we imagine it. It's something we'll explore this year to try to come up with the best way to do it.

kamranjon 6y ago

How does this compare to RedisGraph?

karimtr 6y ago
Good question :D We haven't spent much time trying out RedisGraph, nor we ran any kind of benchmarks at his point, but from what I can see, I would say that the main differences would: 1. Memgraph is a native GraphDB which means that it's specifically engineered to support graph and graph only whereas RedisGraph is a module that runs on top of key-value store (nothing wrong with that, but it might have some limitation when it comes to performance on complex graph algorithms and traversals) 2. Cypher coverage – We cover about 80% of the cypher query language, where I think RedisGraph covers only a small portion (But I hear that they're improving on that) Apart from that, I guess you would really need to test both when for specific use-cases to judge performance and scalability. We'll try to provide some benchmarks in the next few weeks. I hope this helps :D Have you used RedisGraph? Any thoughts?

mralj 6y ago

This looks really cool, congrats on publishing DB :D

mbuda 6y ago
Thank you!

wiradikusuma 6y ago

How does this compare to Dgraph? Most recent discussion about it: https://news.ycombinator.com/item?id=23031762 https://news.ycombinator.com/item?id=23031762 Also, does it support GraphQL?

deleted 6y ago
[deleted]
karimtr 6y ago
To answer your first question, the main difference between Memgraph and Dgraph seem to be: 1. The graph model: Property graph for Memgraph vs RDF for DGraph (Depending on the use-case you have one model might be better than the other. E.g ontologies are better served with RDF, whereas graphs with a lot of properties and labels are better served by the property graph model) 2. Memgraph is an in-memory first system where is Dgraph is a disk-based system. Again depending on your use-case and the performance you are looking for, an in-memory system might be better suited. 3. Query language & Ecosystem: we support Cypher and the Bolt Protocol (Same as Neo4j) so we work with a lot of the existing graph tools. In terms of performance, we don't have official benchmarks yet but we have a few clients that test Memgraph and Dgraph and reported a 3-5x in read performance and about an 8x in write performance for their specific workloads. I hope this answers your question.
- karimtr 6y ago
  Sorry, I missed the second part of you question. You should be able to use the Neo4j GraphQL with Memgraph although we didn't fully test it yet(https://github.com/neo4j-graphql/neo4j-graphql https://github.com/neo4j-graphql/neo4j-graphql).
- micheldiz 6y ago
  Hi guys, I'm community support at Dgraph. Reading that question. I come to clarify some points. First of all, I don't particularly know Memgraph internally. So, I'll stick to Dgraph and related only. About Karimtr's reply 1. The graph model: Although Dgraph uses RDF as input data. Dgraph is not a Triple Store per se - and the RDF we have is a customized version, which means that it is not 100% compatible with any RDF model (e.g. Turtle RDF) - But eventually, many RDF syntaxes may be compatible. The decision to use RDF was made a long time ago, for reasons that I am particularly unaware of. It was long before I joined Dgraph. We also accept JSON as data input. By the way, I also don't understand why Neo4j uses CSV as input data since it is not a Graph standard. RDF itself would be more acceptable than CSV. I assume they use CSV for strategic reasons. 1.1 In practice Dgraph is technically a "Property graph" like. There are no fundamental differences between the Dgraph's graph model with Neo4j other than the language itself and the way the data is stored and injected. 1.2 In Dgraph the data is stored in KV using BadgerDB. 1.3 Ontologies can be represented in any GraphDB. The difference is that Triple Store DBs have a language created to infer data specifically with the concept of ontology. And Triple Stores has standardized data input for this. 2. That's right. However, I think we will soon have the option to keep it in memory. But I don't particularly know how useful this is. Today you can keep some Memory first data, but with the guarantee that they will be saved on disks. This helps in performance when there is no use of NVMe. 3. We have GraphQL+- which is a rich language and inspired by GraphQL. And we also have GraphQL which is an "API" language that is now native in Dgraph. A friendly front-end language. And it works "out of the box" once you mount your schema. Dgraph creates a CRUD model based on your Schema. This reduces production time for your application and less logic on your business side. We are still adding "Black Magic" so that the experience in producing APPs is exceptional. And less code typing. About performance. I suggest doing a test against ludicrous mode https://discuss.dgraph.io/t/sharing-some-numbers-from-the-ludicrous-mode/6310?u=micheldiz https://discuss.dgraph.io/t/sharing-some-numbers-from-the-lu... Cheers.

VRomanov89 6y ago

How do you handle high-availability? Also, do you have some sort of persistency model for the community version which doesn't have HA?

dtomicevic 6y ago
Dominik from Memgraph here. Good questions! All editions of Memgraph persist data to the disk via write-ahead logging (WAL) and periodic snapshots for log compaction so even though Memgraph is designed to be in-memory first, data is always backed up to disk. If you enable asynchronous (periodic fsync) WAL, you can trade off a small window of durability for better performance. Before 1.0, Memgraph leveraged the RAFT consensus algorithm for HA which worked great but had some performance implications. Based on feedback from our users and customers, we have decided to switch our HA implementation to the streaming replication model (similar to PostgreSQL) with automatic failover.
- greenrobot 6y ago
  Interesting. Would you mind to go into details about dropping RAFT? It would make a great tech blog post imho, but maybe you could share some high level insights on that? Related to that: have you considered extensions to the RAFT standard e.g. pipelining?

bavell 6y ago

Congrats on reaching 1.0! I'm always interested in new GraphDB solutions and glad to see the Cypher integration and Neo4j wire compatibility here. I'm excited seeing the progress being made in this space recently (this, redisgraph, etc) - the wider ecosystem is still immature but promising.

dtomicevic 6y ago
Thank you! We are here to do our fair share and provide the community with the right tools to solve different kinds of graph problems. Exciting space nevertheless and growing fast!

bkubicka 6y ago

Congrats on the launch! Which graph algorithms do you support?

mbuda 6y ago
Thank you! Memgraph has built-in BFS, DFS, and weighted-shortest path. These can leverage the query planning (use the details about data distribution to perform better). Since query modules are introduced, algorithms like Page Rank could be easily implemented. Memgraph offers a couple of them out of the box. Last but not least, all algorithms from the Python eco-system could be run inside Memgraph.

abuteau 6y ago

Congrats on the launch! Do you have any plans to release a NodeJs Driver?

mbuda 6y ago
Thank you! So far, Memgraph offers and supports C/C++ library (https://github.com/memgraph/mgclient https://github.com/memgraph/mgclient) and Python binding (https://github.com/memgraph/pymgclient https://github.com/memgraph/pymgclient). Node.js is going to come shortly. Please stay tuned!

bionhoward 6y ago

That cloud pricing page is totally useless because it provides zero detail on what you get for your money, how much you need for a given use case (M8? M32?), and forces you to calculate monthly cost yourself. Further, without any autoscaling/Serverless option, we wind up paying for, polluting for, and wasting energy for 24/7 instances at fixed capacity. Cool product, but the deal is unclear and inflexible, so DynamoDB or Postgres are still better options When is someone going to provide an actual serverless graph db? It’s unbelievable how much masturbatory self-congratulation goes on in the graph database community while Serverless Postgres and Dynamo are exponentially cheaper and better. Is Cypher really worth a 100x-1000x price hike?

FShieldheart 6y ago

Congratulations on your release! Keep up the good work!

dtomicevic 6y ago
Thank you!

JPKab 6y ago

"An integrated ecosystem that would allow data scientists to easily leverage existing data science and machine learning tools to build graph-powered applications with minimum friction." Graph databases have as much to do with data science as they do with any software applications. Going to guess a VC made them put that there. Definitely excited to try it out for a project I'm working on with DAG optimizations.

RocketSyntax 6y ago

This is huge! I'd like to see the roadmap for graph algorithms and graph neural nets! I can provide feedback on both.

mbuda 6y ago
Yea, we're still discussing the exact roadmap on this side. In the short term, the plan is to integrate better with the whole ecosystem. We are still exploring options on the graph NN side. Do you have any specific algorithms to point out? :D

zvone 6y ago

Hey everyone, Two questions, please. What about C#? Do you have some API? What about spatial search? Do you have that integrated in API? Thanks

bfrit 6y ago

This is awesome!