4 ms·

You should add the Neo4j (http://neo4j.org/ http://neo4j.org/) graph database to the list. Graphs are a much more modern and elegant way of storing relational

by espeed 15y ago

You should add the Neo4j (http://neo4j.org/ http://neo4j.org/) graph database to the list. Graphs are a much more modern and elegant way of storing relational data. I've used Postgres for over 10 years, but it's not a graph database. With graph databases you don't have to mess with tables or joins -- everything is implicitly joined. And Neo4j is ridiculously sweet -- store 32 billion nodes (http://blog.neo4j.org/2011/03/neo4j-13-abisko-lampa-m04-size-really.html http://blog.neo4j.org/2011/03/neo4j-13-abisko-lampa-m04-size...) with 2 million traversals per second (http://www.infoq.com/news/2010/02/neo4j-10 http://www.infoq.com/news/2010/02/neo4j-10), and you can use Gremlin with it (the graph traversal language), which let's you calculate PageRank in 2 lines. Neo4j is open source, and the Community Edition is now free (https://github.com/neo4j/community https://github.com/neo4j/community). I recommend pairing it with the TinkerPop stack (http://www.tinkerpop.com/ http://www.tinkerpop.com/).

deleted 15y ago
[deleted]
versae 15y ago
We are also using Neo4j in an academic environment and we are really pleased with it. We recommend it despite of the lack of a good native binding Python, we had to build a neo4j-rest-client (https://github.com/versae/neo4j-rest-client https://github.com/versae/neo4j-rest-client) through REST API.
- espeed 15y ago
  There is an open-source Python persistence framework in the works called Bulbs that connects to Neo4j through the Rexster REST server, and there are binary bindings in the works as well. There is also a Python open-source Web development framework for graph databases called Bulbflow that is based on Bulbs and Flask. Both frameworks should be released in the next few weeks.
jacques_chester 15y ago
> Graphs are a much more modern and elegant way of storing relational data. Actually, storing data as graphs is older than relational approaches. It used to be called "network databases". They were not supplanted for the hell of it, relational databases have certain advantages. For select applications, object databases are absolutely the way to go. But for most purposes relational is hard to beat.
- espeed 15y ago
  For tabular data, relational databases rock. But the relational model doesn't align well with object-orientated programming so you have an ORM layer that adds complexity to your code. And with relational databases, the complexity of your schema grows with the complexity of the data. The graph-database model simplifies much of this and makes working with the modern-day social graph so much cleaner. Graphs allow you to do powerful things like find inferences inside the data in ways that would be hard to do with relational databases. How would you calculate PageRank using a relational database? As I said, with a graph database and Gremlin, you can do it in 2 lines. To see the types of things you can do with graphs, check out Marko's short screencast on Gremlin (http://www.youtube.com/watch?v=5wpTtEBK4-E http://www.youtube.com/watch?v=5wpTtEBK4-E). And also check out Peter Neubauer's introduction to graph databases and how they compare to RDBMS' and where they stand in the NOSQL-movement (http://www.infoq.com/articles/graph-nosql-neo4j http://www.infoq.com/articles/graph-nosql-neo4j).
  - jacques_chester 15y ago
    > But the relational model doesn't align well with object-orientated programming so you have an ORM layer that adds complexity to your code. The mismatch comes about for a number of reasons: * OOP has no formal basis so it can't be reliably transformed into relational terms. * OOP is identity-bound -- each object is essentially an address in memory, not a relation in a set. * The biggie: OOP mixes data with behaviour. Relational does not. > How would you calculate PageRank using a relational database? As I said, with a graph database and Gremlin, you can do it in 2 lines. For something on PageRank's scale, a custom datastore based on matrices and their multiplication makes business sense. Or MapReduce over a distributed key-value store (note that these are both OLAP approaches). Still. SQL's a bit verbose, but these days we have recursive queries. For Oracle users, I'm talking about CONNECT BY. If I find myself running the social graph every minute, I develop an ETL package that periodically moves data from my write-bound system to my read-bound system with a more query-friendly schema. Depending on how you look on it, relational systems invented "eventual consistency". More to the point, boring old database greybeards have learnt that OLTP and OLAP are very different use cases. The 3/4/5NF of the OLTP database will be very different from the star schema of the OLAP database. There's really not much about NoSQL that hasn't already been done, under a different name, by the relational crowd. I still think there is a place for NoSQL. It's just not as universal a replacement for relational systems as many make it out to be, whether we're talking about document stores, distributed key-value stores, graph stores and so on.
gaius 15y ago
Sorry, but this is very amusing: IBM were doing this, with IMS, in the 1960s and IMS is still their highest-revenue product today.
- espeed 15y ago
  But isn't IMB a hierarchical database (http://en.wikipedia.org/wiki/Hierarchical_database_model http://en.wikipedia.org/wiki/Hierarchical_database_model), which is a tree-like structure? This is somewhat different than a general graph database (http://en.wikipedia.org/wiki/Graph_database http://en.wikipedia.org/wiki/Graph_database) where nodes are not restricted to being a hierarchy.

by espeed 15y ago

deleted 15y ago

[deleted]

versae 15y ago

We are also using Neo4j in an academic environment and we are really pleased with it. We recommend it despite of the lack of a good native binding Python, we had to build a neo4j-rest-client (https://github.com/versae/neo4j-rest-client https://github.com/versae/neo4j-rest-client) through REST API.

espeed 15y ago
There is an open-source Python persistence framework in the works called Bulbs that connects to Neo4j through the Rexster REST server, and there are binary bindings in the works as well. There is also a Python open-source Web development framework for graph databases called Bulbflow that is based on Bulbs and Flask. Both frameworks should be released in the next few weeks.

jacques_chester 15y ago

> Graphs are a much more modern and elegant way of storing relational data. Actually, storing data as graphs is older than relational approaches. It used to be called "network databases". They were not supplanted for the hell of it, relational databases have certain advantages. For select applications, object databases are absolutely the way to go. But for most purposes relational is hard to beat.

espeed 15y ago
For tabular data, relational databases rock. But the relational model doesn't align well with object-orientated programming so you have an ORM layer that adds complexity to your code. And with relational databases, the complexity of your schema grows with the complexity of the data. The graph-database model simplifies much of this and makes working with the modern-day social graph so much cleaner. Graphs allow you to do powerful things like find inferences inside the data in ways that would be hard to do with relational databases. How would you calculate PageRank using a relational database? As I said, with a graph database and Gremlin, you can do it in 2 lines. To see the types of things you can do with graphs, check out Marko's short screencast on Gremlin (http://www.youtube.com/watch?v=5wpTtEBK4-E http://www.youtube.com/watch?v=5wpTtEBK4-E). And also check out Peter Neubauer's introduction to graph databases and how they compare to RDBMS' and where they stand in the NOSQL-movement (http://www.infoq.com/articles/graph-nosql-neo4j http://www.infoq.com/articles/graph-nosql-neo4j).
- jacques_chester 15y ago
  > But the relational model doesn't align well with object-orientated programming so you have an ORM layer that adds complexity to your code. The mismatch comes about for a number of reasons: * OOP has no formal basis so it can't be reliably transformed into relational terms. * OOP is identity-bound -- each object is essentially an address in memory, not a relation in a set. * The biggie: OOP mixes data with behaviour. Relational does not. > How would you calculate PageRank using a relational database? As I said, with a graph database and Gremlin, you can do it in 2 lines. For something on PageRank's scale, a custom datastore based on matrices and their multiplication makes business sense. Or MapReduce over a distributed key-value store (note that these are both OLAP approaches). Still. SQL's a bit verbose, but these days we have recursive queries. For Oracle users, I'm talking about CONNECT BY. If I find myself running the social graph every minute, I develop an ETL package that periodically moves data from my write-bound system to my read-bound system with a more query-friendly schema. Depending on how you look on it, relational systems invented "eventual consistency". More to the point, boring old database greybeards have learnt that OLTP and OLAP are very different use cases. The 3/4/5NF of the OLTP database will be very different from the star schema of the OLAP database. There's really not much about NoSQL that hasn't already been done, under a different name, by the relational crowd. I still think there is a place for NoSQL. It's just not as universal a replacement for relational systems as many make it out to be, whether we're talking about document stores, distributed key-value stores, graph stores and so on.

gaius 15y ago

Sorry, but this is very amusing: IBM were doing this, with IMS, in the 1960s and IMS is still their highest-revenue product today.

espeed 15y ago
But isn't IMB a hierarchical database (http://en.wikipedia.org/wiki/Hierarchical_database_model http://en.wikipedia.org/wiki/Hierarchical_database_model), which is a tree-like structure? This is somewhat different than a general graph database (http://en.wikipedia.org/wiki/Graph_database http://en.wikipedia.org/wiki/Graph_database) where nodes are not restricted to being a hierarchy.