5 ms·

Dgraph, GraphQL, Schemas, & CRUD – Ardan Labs

63 points by ncputo 6y ago

vanilla-almond 6y ago
GraphQL seems to sit between the strict relational model and the more free-form approach of NoSQL databases (from I can see, but correct me if I am wrong). I have not used a GraphQL database, but one of the things I fear about designing a relational database is mentioned in the article: >"Relational databases are very rigid so you need to really make sure you understand your data upfront. Changing the database is a big effort..." (emphasis is mine) Given how often requirements change in a business, it seems impossible to anticipate all future requirements at the outset of designing a database. Is it true that changing a (relational) database schema is a big effort? Is this a strong case for GraphQL?
- joestrouth1 6y ago
  GraphQL is not a database technology. It's a Query Language that can sit in front of REST APIs, databases of all kinds, or in-memory data. As such it typically has no impact on your database schema or any difficulties associated with it. It can be true that changing a relational db schema is a big effort and it can be true that graph (not GraphQL) dbs are more flexible. Even for neo4j and JanusGraph, which don't offer GraphQL by default. They're NoSQL databases that store graphs instead of documents or k/v pairs. It just happens that Dgraph decided to integrate it
- brentjanderson 6y ago
  As always, it depends. When requirements change, there are a few things that change altogether - and only some of them should affect the database: 1. The data at rest (in your relational database) - database migrations are the solution to this problem, in one way or another. Migrations are well understood, and are not hard - just tricky to get right depending on the complexity of the change. 2. Rules about how to turn that persisted data into something you can reason about at runtime (aka hydration logic) 3. Rules about how to change data from one form or another due to business rules (aka business logic) If your database is an implementation detail and not the heart of how you reason about your system, then the difficulty in making the migration from one form of data at rest to another should be easier. If you rely on your database for storage and business logic, then you may have a more difficult time.
- mrjn 6y ago
  (author of Dgraph here) There're a bunch of changes you can do in Dgraph, that're very convenient. For e.g., going from 1:1 relationship to 1:many relationships (see my comment about movies and directors). Having sparse data models and such. Then, able to change from int to float, to date -- all these constitute a flexible schema that something like Dgraph is built around, and an relational DB is not -- which all make it easier to iterate upon data models, way better than what relational tables can give.
- claytonjy 6y ago
  It doesn't have to be, but tooling and initial design matters a lot. I've done a lot of significant migrations this year (changing schemas of hundreds of millions of rows in postgres) in a downtime-free, non-breaking way, but there were two big pieces: - separate internal schemas from external ones, so you can change the internals while maintaining your interface contract. Your database has an API whether you realize it or not, so treat it like one. I literally have schemas like "api" and "api_v2". - use a great migration tool. I've been using sqitch, which I love because it encourages me to think about how to _verify_ each migration does the correct thing. Tie that into CI/CD in a nice way and DB migrations are no longer scary That probably downplays the effort; as someone who's never done DBA work before, I've had to learn a lot about PL/pgSQL, transaction isolation levels, role management, etc. for the first time. I think I'm better for it, though, and I've quite enjoyed going beyond query-writing.
mamcx 6y ago
>Relational databases are very rigid so you need to really make sure you understand your data upfront. Is NOT true: CREATE TABLE/VIEW ... ALTER TABLE ... SELECT (whatever) ... Changing a rdbms at runtime is fully supported, in MUCH better ways than almost all "nosql" databases. > Changing the database is a big effort Is much less effort than nosql. With more guaranties than the change is CORRECT, and not left you half documents in one version and the other in other version. > These databases are hard to distribute and scale so you tend to end up with single instances that are very large and require replication for backup. The cloud providers have relational databases today that are supposed to scale, but I have no experience with them. Is very easy to be "web scale" if you don't care about data integrity. Very few nosql are half good about this. Scaling a rdbms is very similar to have a less brittle nosql product (like mongo), is about how you model the data. I was in a team behind one of the largest deployments on google cloud store. The amount of coding for cover for the store lack of integrity was a big part of the codebase. Despite the supposedly massive amount of data I don't see why have a rdbms in the back could have been worse (when anyway you must put in front redis and other caches for high ingestion). I think for the majority of the startup deployments, rarely a non-rdbms make much sense, specially if the team have not idea of what truly a good rdbms can do, or how code by hand a correct transaction code...
- mrjn 6y ago
  (author of Dgraph here) I'd argue relational DBs get way more credit than they're worth. And it is indeed tricky to make them work across different data models -- that's by now an established fact across big SV companies. Restricting direct access to rigid RDBMs and making data modeling easier via graphs is why Facebook Tao, Dropbox Edgestore, Airbnb Knowledge Graph and so on exist. See my post [1]. I'll give you a simple example. Consider, movies and directors. If you started with the fact that each movie has exactly one director, you'd design exactly one table, with a director column. Later, if you realize that each movie can have multiple directors, you can't just alter that table. You need two new tables -- one for directors, and another one to connect movies to directors. That's not simple, that's a big change which requires a bunch of work. Doing this in graphs is simple. Adding a bunch more outward relationships is simple. Data modeling, in general, is way simpler in graphs than in RDBMs. Mongo and such came out of the Bigtable era. Cockroach and Dgraph are coming out of the Spanner era -- where it's not only about horizontal scalability, but also about strong data integrity and distributed ACID transactions. [1]: https://dgraph.io/blog/post/how-dgraph-labs-raised-series-a/ https://dgraph.io/blog/post/how-dgraph-labs-raised-series-a/

63 points by ncputo 6y ago

GraphQL seems to sit between the strict relational model and the more free-form approach of NoSQL databases (from I can see, but correct me if I am wrong). I have not used a GraphQL database, but one of the things I fear about designing a relational database is mentioned in the article: >"Relational databases are very rigid so you need to really make sure you understand your data upfront. Changing the database is a big effort..." (emphasis is mine) Given how often requirements change in a business, it seems impossible to anticipate all future requirements at the outset of designing a database. Is it true that changing a (relational) database schema is a big effort? Is this a strong case for GraphQL?

joestrouth1 6y ago
GraphQL is not a database technology. It's a Query Language that can sit in front of REST APIs, databases of all kinds, or in-memory data. As such it typically has no impact on your database schema or any difficulties associated with it. It can be true that changing a relational db schema is a big effort and it can be true that graph (not GraphQL) dbs are more flexible. Even for neo4j and JanusGraph, which don't offer GraphQL by default. They're NoSQL databases that store graphs instead of documents or k/v pairs. It just happens that Dgraph decided to integrate it
brentjanderson 6y ago
As always, it depends. When requirements change, there are a few things that change altogether - and only some of them should affect the database: 1. The data at rest (in your relational database) - database migrations are the solution to this problem, in one way or another. Migrations are well understood, and are not hard - just tricky to get right depending on the complexity of the change. 2. Rules about how to turn that persisted data into something you can reason about at runtime (aka hydration logic) 3. Rules about how to change data from one form or another due to business rules (aka business logic) If your database is an implementation detail and not the heart of how you reason about your system, then the difficulty in making the migration from one form of data at rest to another should be easier. If you rely on your database for storage and business logic, then you may have a more difficult time.
mrjn 6y ago
(author of Dgraph here) There're a bunch of changes you can do in Dgraph, that're very convenient. For e.g., going from 1:1 relationship to 1:many relationships (see my comment about movies and directors). Having sparse data models and such. Then, able to change from int to float, to date -- all these constitute a flexible schema that something like Dgraph is built around, and an relational DB is not -- which all make it easier to iterate upon data models, way better than what relational tables can give.
claytonjy 6y ago
It doesn't have to be, but tooling and initial design matters a lot. I've done a lot of significant migrations this year (changing schemas of hundreds of millions of rows in postgres) in a downtime-free, non-breaking way, but there were two big pieces: - separate internal schemas from external ones, so you can change the internals while maintaining your interface contract. Your database has an API whether you realize it or not, so treat it like one. I literally have schemas like "api" and "api_v2". - use a great migration tool. I've been using sqitch, which I love because it encourages me to think about how to _verify_ each migration does the correct thing. Tie that into CI/CD in a nice way and DB migrations are no longer scary That probably downplays the effort; as someone who's never done DBA work before, I've had to learn a lot about PL/pgSQL, transaction isolation levels, role management, etc. for the first time. I think I'm better for it, though, and I've quite enjoyed going beyond query-writing.

mamcx 6y ago

>Relational databases are very rigid so you need to really make sure you understand your data upfront. Is NOT true: CREATE TABLE/VIEW ... ALTER TABLE ... SELECT (whatever) ... Changing a rdbms at runtime is fully supported, in MUCH better ways than almost all "nosql" databases. > Changing the database is a big effort Is much less effort than nosql. With more guaranties than the change is CORRECT, and not left you half documents in one version and the other in other version. > These databases are hard to distribute and scale so you tend to end up with single instances that are very large and require replication for backup. The cloud providers have relational databases today that are supposed to scale, but I have no experience with them. Is very easy to be "web scale" if you don't care about data integrity. Very few nosql are half good about this. Scaling a rdbms is very similar to have a less brittle nosql product (like mongo), is about how you model the data. I was in a team behind one of the largest deployments on google cloud store. The amount of coding for cover for the store lack of integrity was a big part of the codebase. Despite the supposedly massive amount of data I don't see why have a rdbms in the back could have been worse (when anyway you must put in front redis and other caches for high ingestion). I think for the majority of the startup deployments, rarely a non-rdbms make much sense, specially if the team have not idea of what truly a good rdbms can do, or how code by hand a correct transaction code...

mrjn 6y ago
(author of Dgraph here) I'd argue relational DBs get way more credit than they're worth. And it is indeed tricky to make them work across different data models -- that's by now an established fact across big SV companies. Restricting direct access to rigid RDBMs and making data modeling easier via graphs is why Facebook Tao, Dropbox Edgestore, Airbnb Knowledge Graph and so on exist. See my post [1]. I'll give you a simple example. Consider, movies and directors. If you started with the fact that each movie has exactly one director, you'd design exactly one table, with a director column. Later, if you realize that each movie can have multiple directors, you can't just alter that table. You need two new tables -- one for directors, and another one to connect movies to directors. That's not simple, that's a big change which requires a bunch of work. Doing this in graphs is simple. Adding a bunch more outward relationships is simple. Data modeling, in general, is way simpler in graphs than in RDBMs. Mongo and such came out of the Bigtable era. Cockroach and Dgraph are coming out of the Spanner era -- where it's not only about horizontal scalability, but also about strong data integrity and distributed ACID transactions. [1]: https://dgraph.io/blog/post/how-dgraph-labs-raised-series-a/ https://dgraph.io/blog/post/how-dgraph-labs-raised-series-a/