4 ms·

(author of Dgraph here) I'd argue relational DBs get way more credit than they're worth. And it is indeed tricky to make them work across different data models

by mrjn 6y ago

(author of Dgraph here) I'd argue relational DBs get way more credit than they're worth. And it is indeed tricky to make them work across different data models -- that's by now an established fact across big SV companies. Restricting direct access to rigid RDBMs and making data modeling easier via graphs is why Facebook Tao, Dropbox Edgestore, Airbnb Knowledge Graph and so on exist. See my post [1]. I'll give you a simple example. Consider, movies and directors. If you started with the fact that each movie has exactly one director, you'd design exactly one table, with a director column. Later, if you realize that each movie can have multiple directors, you can't just alter that table. You need two new tables -- one for directors, and another one to connect movies to directors. That's not simple, that's a big change which requires a bunch of work. Doing this in graphs is simple. Adding a bunch more outward relationships is simple. Data modeling, in general, is way simpler in graphs than in RDBMs. Mongo and such came out of the Bigtable era. Cockroach and Dgraph are coming out of the Spanner era -- where it's not only about horizontal scalability, but also about strong data integrity and distributed ACID transactions. [1]: https://dgraph.io/blog/post/how-dgraph-labs-raised-series-a/ https://dgraph.io/blog/post/how-dgraph-labs-raised-series-a/

simonw 6y ago
> That's not simple, that's a big change which requires a bunch of work. I feel like many of these criticisms of RDBMS come from a perspective of not working with a great database migrations system. I've made changes like this (one-to-many switched for many-to-many) using Django migrations many times over, and I find it easy. But that's because I've built up the experience with the tooling over time. Learning migration tooling in this way feels like a much smaller lift to me than entirely switching to a graph database. (I'm a big fan of Dgraph, but I've not yet found the right project to apply it. I'm not interested in it as a replacement for RDBMS, I want to use it to solve graph problems that are genuinely painful with a relational database. Easier schema changes aren't that for me.)
- mrjn 6y ago
  Hey Simon, Totally see your perspective. You're already familiar with data migration, but not familiar with graph DBs. The former is just easier to do than to invest in the learning curve involved with the latter. Valid point. My take is more root-cause and principle based. If we were to think of a different way to build a DB, then avoiding a data migration altogether is a better approach. Avoiding the downtime, extra work, extra code and so on is just better for the user. Moreover, simpler data models, avoiding data duplication and such become very convincing features. It's also not so interesting to me when graph DBs are only applied to specific graph problems, like Page Rank and such. We didn't build Dgraph just to solve Page Rank. The reason we built Dgraph was because we saw the complex data modeling problem Google Knowledge Graph threw at us. Graph system allowed that problem to be solved in a really clean and simple way. And it's the same problem we see other companies dealing with as well.
default-kramer 6y ago
In your movie+director example, changing a relational database is relatively easy. Yes, you create two new tables but that is not hard. And you relocate the director data from the movie table into the new director table, also not hard. The hard part is updating all the application code that was written assuming that a movie has one director. I'm not super familiar with graph DBs, but I don't see how they could possibly help with that.
- mrjn 6y ago
  Add some producers, cinematographers, actors, oscar awards and so on. Complexity adds up. OTOH, In Dgraph GraphQL, you'd just do a simple schema edit from: type Movie { ... director: Director } to type Movie { ... directors: [Director] } You're right about application code stuff. But, that's a common cost across both the spaces -- though, I'd argue that JSON responses back from GraphQL (/ Dgraph) make it easier.
  - alexchamberlain 6y ago
    I'm a massive proponent of the graph model, but the the RDBMS fans have it right here: the schema change is trivial in both cases; the hard part is managing the roll out of those changes to prevent outage on the application you are actually selling.
mamcx 6y ago
> Later, if you realize that each movie can have multiple directors This still require change across the codebase, where the db migration is fairly, the most simply step. But: Is similar to "static VS dynamic" where the change in rdbms come BEFORE the new data and in most nosql AFTER. If somehow the data come in a "progressive do changes pls" then is clear the way most db schemas are, is harder with BEFORE. So, yeah, exist places that a specialized store make more sense. But just say "is harder" invite this critique, and worse, in the minds of inexperience developers (that are the ones that deploy blind) could lead to "get out of a rdbms, are old and bad!", when in fact is much better, most of time. ALSO: > Restricting direct access to rigid RDBMs and making data modeling easier via graphs is why.... The relational model could handle that. BUT, the implementation inside rdbms is just a subset of the full potential. One example: You can't embed tables in tables. And relational/sql engine have weak support for hierarchies and graph, despite the fact you see sql on top of other non-relational engines. I'm building a relational lang in the spare time, and without the constrain of fit the(traditional) rdbms is clear the db guys need to up their game!
lmeyerov 6y ago
From a user perspective, what we find the graph dynamism is more important for Self-Serve Analytics, like a fast-moving data science or analyst or dev team, than for DBAs. So a diff perspective: DBA SCHEMA CHANGE - Relational: Admins alter the schema and devs maybe run a migration - Graph: Projects start untyped, but almost all Graphistry has helped are effectively & ultimately typed & locked. So ends up working same as relational. (Ex: ProjectDomino.org took just maybe 1mo before the neo4j DBA locked the schema to avoid inadvertent value pollution.) DYNAMIC QUERIES Once a bunch of entity types and relationships get involved, you build up a zoo of tables. This is painful for adding data ("new dataset X needs schema Y, or subtypes into existing table Z by dynamic type field A + generic data stuffed into field metadata"), and much worse, given some data, trying to figure out how it indexes across 20+ tables. Three examples: an Infosec Jupyterthon talk was about how to make sense of OSQuery's ~100 different tables for OS logs, a genetics friend got onboarded to a DB with 30 tables & views they know nothing about, and in Project Domino, you can do all sorts of things with URLs and Topics across our datasets & analyses. Relational is great for a rigid CRUD app. But once you hit analytics, you get swamped with data integration tasks (discovery, linking, fusion, ...), and tables become a PITA. Formally, graph queries like ("everything 1 step out") gives you genericity in quantification where SQL forces you to know the exact table name & field name ahead of time.
tuatoru 6y ago
> I'll give you a simple example. Consider, movies and directors. If you started with the fact that each movie has exactly one director, you'd design exactly one table, with a director column. With respect, if you have that little domain knowledge, you shouldn't be designing--er, coding--anything other than a throwaway exploratory scrap. And in preference to doing that, you should be getting out and talking to the people who plan to use your database, and learning how they think, and what about. Systems analysis is another '80s development "fad" that is way underrated.
- xupybd 6y ago
  Businesses change very quickly. To say that you shouldn't code until you know the domain well enough to model it perfectly is not wise. You build software to solve real world problems. Sometimes that means reworking the solution as the problem changes.

by mrjn 6y ago

simonw 6y ago

> That's not simple, that's a big change which requires a bunch of work. I feel like many of these criticisms of RDBMS come from a perspective of not working with a great database migrations system. I've made changes like this (one-to-many switched for many-to-many) using Django migrations many times over, and I find it easy. But that's because I've built up the experience with the tooling over time. Learning migration tooling in this way feels like a much smaller lift to me than entirely switching to a graph database. (I'm a big fan of Dgraph, but I've not yet found the right project to apply it. I'm not interested in it as a replacement for RDBMS, I want to use it to solve graph problems that are genuinely painful with a relational database. Easier schema changes aren't that for me.)

mrjn 6y ago
Hey Simon, Totally see your perspective. You're already familiar with data migration, but not familiar with graph DBs. The former is just easier to do than to invest in the learning curve involved with the latter. Valid point. My take is more root-cause and principle based. If we were to think of a different way to build a DB, then avoiding a data migration altogether is a better approach. Avoiding the downtime, extra work, extra code and so on is just better for the user. Moreover, simpler data models, avoiding data duplication and such become very convincing features. It's also not so interesting to me when graph DBs are only applied to specific graph problems, like Page Rank and such. We didn't build Dgraph just to solve Page Rank. The reason we built Dgraph was because we saw the complex data modeling problem Google Knowledge Graph threw at us. Graph system allowed that problem to be solved in a really clean and simple way. And it's the same problem we see other companies dealing with as well.

default-kramer 6y ago

In your movie+director example, changing a relational database is relatively easy. Yes, you create two new tables but that is not hard. And you relocate the director data from the movie table into the new director table, also not hard. The hard part is updating all the application code that was written assuming that a movie has one director. I'm not super familiar with graph DBs, but I don't see how they could possibly help with that.

mrjn 6y ago
Add some producers, cinematographers, actors, oscar awards and so on. Complexity adds up. OTOH, In Dgraph GraphQL, you'd just do a simple schema edit from: type Movie { ... director: Director } to type Movie { ... directors: [Director] } You're right about application code stuff. But, that's a common cost across both the spaces -- though, I'd argue that JSON responses back from GraphQL (/ Dgraph) make it easier.
- alexchamberlain 6y ago
  I'm a massive proponent of the graph model, but the the RDBMS fans have it right here: the schema change is trivial in both cases; the hard part is managing the roll out of those changes to prevent outage on the application you are actually selling.

mamcx 6y ago

> Later, if you realize that each movie can have multiple directors This still require change across the codebase, where the db migration is fairly, the most simply step. But: Is similar to "static VS dynamic" where the change in rdbms come BEFORE the new data and in most nosql AFTER. If somehow the data come in a "progressive do changes pls" then is clear the way most db schemas are, is harder with BEFORE. So, yeah, exist places that a specialized store make more sense. But just say "is harder" invite this critique, and worse, in the minds of inexperience developers (that are the ones that deploy blind) could lead to "get out of a rdbms, are old and bad!", when in fact is much better, most of time. ALSO: > Restricting direct access to rigid RDBMs and making data modeling easier via graphs is why.... The relational model could handle that. BUT, the implementation inside rdbms is just a subset of the full potential. One example: You can't embed tables in tables. And relational/sql engine have weak support for hierarchies and graph, despite the fact you see sql on top of other non-relational engines. I'm building a relational lang in the spare time, and without the constrain of fit the(traditional) rdbms is clear the db guys need to up their game!

lmeyerov 6y ago

From a user perspective, what we find the graph dynamism is more important for Self-Serve Analytics, like a fast-moving data science or analyst or dev team, than for DBAs. So a diff perspective: DBA SCHEMA CHANGE - Relational: Admins alter the schema and devs maybe run a migration - Graph: Projects start untyped, but almost all Graphistry has helped are effectively & ultimately typed & locked. So ends up working same as relational. (Ex: ProjectDomino.org took just maybe 1mo before the neo4j DBA locked the schema to avoid inadvertent value pollution.) DYNAMIC QUERIES Once a bunch of entity types and relationships get involved, you build up a zoo of tables. This is painful for adding data ("new dataset X needs schema Y, or subtypes into existing table Z by dynamic type field A + generic data stuffed into field metadata"), and much worse, given some data, trying to figure out how it indexes across 20+ tables. Three examples: an Infosec Jupyterthon talk was about how to make sense of OSQuery's ~100 different tables for OS logs, a genetics friend got onboarded to a DB with 30 tables & views they know nothing about, and in Project Domino, you can do all sorts of things with URLs and Topics across our datasets & analyses. Relational is great for a rigid CRUD app. But once you hit analytics, you get swamped with data integration tasks (discovery, linking, fusion, ...), and tables become a PITA. Formally, graph queries like ("everything 1 step out") gives you genericity in quantification where SQL forces you to know the exact table name & field name ahead of time.

tuatoru 6y ago

> I'll give you a simple example. Consider, movies and directors. If you started with the fact that each movie has exactly one director, you'd design exactly one table, with a director column. With respect, if you have that little domain knowledge, you shouldn't be designing--er, coding--anything other than a throwaway exploratory scrap. And in preference to doing that, you should be getting out and talking to the people who plan to use your database, and learning how they think, and what about. Systems analysis is another '80s development "fad" that is way underrated.

xupybd 6y ago
Businesses change very quickly. To say that you shouldn't code until you know the domain well enough to model it perfectly is not wise. You build software to solve real world problems. Sometimes that means reworking the solution as the problem changes.