4 ms·

Like many developers, I've used postgresql unquestioningly for many years, say 10. But if you ask me, I've rarely have had to face stringent scaling or availabi

by vemv 6y ago

Like many developers, I've used postgresql unquestioningly for many years, say 10. But if you ask me, I've rarely have had to face stringent scaling or availability requirements. Many deployments were a variation of AWS RDS or a similarly managed offering for HA. And they weren't exactly flawless. So this article is good food for thought. Why are we using postgresql? Because of the features, developer-friendliness, and so on. Does that align with actual customer requirements? Not necessarily. This problem framing aligns well with other phenomena one can observe. i.e. many complexities in our industry are completely self-inflicted (some personal 'favorites': OOP, microservices, SPAs, graphql). What if postgresql could be added to that list? Note that I'm not necessarily questioning postgresql as a project, but instead, the hype process that can lead communities to unquestioningly adopt this or that product without making an actually informed assessment.

__jem 6y ago
You have to put your data somewhere... are commercial database vendors or nosql solutions that much better?
- vemv 6y ago
  I cannot make an expert assessment myself but for one thing mysql is occasionally picked precisely because of its HA story. It was the case in a couple past jobs of mine, back then I didn't particularly appreciate it but now I might see it with different eyes. Also commercial databases do have a better HA story - at least that's their reputation. The current status quo is that everything should be free, but obviously that has a fundamental contradiction with being a professional software developer in the first place.
  - MaxBarraclough 6y ago
    What do you make of Amazon Aurora?
nicoburns 6y ago
I think Postgres is great for many of the same reasons that people often (wrongly) tout NoSQL systems for. It's flexible, featureful, simple and quick to get started. And unlike most NoSQL systems, it has full ACID compliance and can scale well past MVP stage to the point that most businesses will never hit its limitations. If you do hit really huge scale then you will need to start looking beyond Postgres to solutions like Cassandra, Scylla, etc. But hopefully by that point you have a large dev team capable of handling the extra complexity.
- theshrike79 6y ago
  PostgreSQL is better at being MongoDB than Mongo is. You can just add a JSON column and do queries on the content, index the table on individual values etc.
  - pillefitz 6y ago
    One of the benefits of using Mongo is its horizontal scalability, not necessarily its ability to store documents.
    - jd_mongodb 6y ago
      That is somewhat missing the point of MongoDB. We start with easy to use document storage, querying, indexing, and dynamic schema. We end with horizontal scalability.
  - jd_mongodb 6y ago
    Postgres is a fantastic relational database but for storage, indexing and querying of JSON documents there is no comparison to MongoDB. MongoDB is built from the ground up as a JSON store (actually a binary encoding of JSON called BSON). I would encourage you to try it out I think you would be surprised by what is in the box.
jeltz 6y ago
I wouldn't say postgresql is a self inflicted problem, like eg some argue micro services are, it is rather that choosing PostgreSQL optimizes for developer experience over things like flawless HA and zero downtime upgrades. So the tradeoff is between being able to build products faster and cheaper while sacrificing some reliability.
collyw 6y ago
Do you have any good examples of Graphql increasing complexity (as opposed to REST)? A dev at my work is pushing for it, and I am against it because it's another tech that isn't really solving any problems that we currently have (too much complexity is our number one problem). It would be good to have some examples to show them.
- allochthon 6y ago
  I like to think of GraphQL as a specific tool to address a specific set of challenges. Is it convenient to be able to batch up a bunch of requests in one go for a frontend JS client? Would it be useful to be able to specify a subset of the fields to be returned in a query? If the answers to these questions and others are "yes," GraphQL might be a good fit for what one is doing. I've personally had good experiences with it. It does bring in additional complexity. Is the complexity worth taking on in a specific dev team? This is something that will depend on the team. It is usually the case that a tool is not useful outside of the set of challenges it is tailored to address. And in some cases a tool is not well designed or implemented. It's always good to use the right tool for the job, at the level of complexity that a team is in a good position to handle.
  - gsvclass 6y ago
    You can use GraphJin it automatically compiles GraphQL queries into Postgres SQL ones. The core idea is to simplify frontend development https://github.com/dosco/graphjin https://github.com/dosco/graphjin
  - collyw 6y ago
    I keep hearing that you can specify the only fields that you need, but to be honest that isn't a problem we have. I have only taken a cursory glance, but adding a schema in at the request level, which then needs translated to a Mongo query just seems to be adding complexity rather than removing it. And URL's are going to be ugly if you are using GET requests, one of the things I am pushing for with a REST API. (Currently our API is not RESTful and most requests go over POSTs, giving less information when debugging).

by vemv 6y ago

__jem 6y ago

You have to put your data somewhere... are commercial database vendors or nosql solutions that much better?

vemv 6y ago
I cannot make an expert assessment myself but for one thing mysql is occasionally picked precisely because of its HA story. It was the case in a couple past jobs of mine, back then I didn't particularly appreciate it but now I might see it with different eyes. Also commercial databases do have a better HA story - at least that's their reputation. The current status quo is that everything should be free, but obviously that has a fundamental contradiction with being a professional software developer in the first place.
- MaxBarraclough 6y ago
  What do you make of Amazon Aurora?

nicoburns 6y ago

I think Postgres is great for many of the same reasons that people often (wrongly) tout NoSQL systems for. It's flexible, featureful, simple and quick to get started. And unlike most NoSQL systems, it has full ACID compliance and can scale well past MVP stage to the point that most businesses will never hit its limitations. If you do hit really huge scale then you will need to start looking beyond Postgres to solutions like Cassandra, Scylla, etc. But hopefully by that point you have a large dev team capable of handling the extra complexity.

theshrike79 6y ago
PostgreSQL is better at being MongoDB than Mongo is. You can just add a JSON column and do queries on the content, index the table on individual values etc.
- pillefitz 6y ago
  One of the benefits of using Mongo is its horizontal scalability, not necessarily its ability to store documents.
  - jd_mongodb 6y ago
    That is somewhat missing the point of MongoDB. We start with easy to use document storage, querying, indexing, and dynamic schema. We end with horizontal scalability.
- jd_mongodb 6y ago
  Postgres is a fantastic relational database but for storage, indexing and querying of JSON documents there is no comparison to MongoDB. MongoDB is built from the ground up as a JSON store (actually a binary encoding of JSON called BSON). I would encourage you to try it out I think you would be surprised by what is in the box.

jeltz 6y ago

I wouldn't say postgresql is a self inflicted problem, like eg some argue micro services are, it is rather that choosing PostgreSQL optimizes for developer experience over things like flawless HA and zero downtime upgrades. So the tradeoff is between being able to build products faster and cheaper while sacrificing some reliability.

collyw 6y ago

Do you have any good examples of Graphql increasing complexity (as opposed to REST)? A dev at my work is pushing for it, and I am against it because it's another tech that isn't really solving any problems that we currently have (too much complexity is our number one problem). It would be good to have some examples to show them.

allochthon 6y ago
I like to think of GraphQL as a specific tool to address a specific set of challenges. Is it convenient to be able to batch up a bunch of requests in one go for a frontend JS client? Would it be useful to be able to specify a subset of the fields to be returned in a query? If the answers to these questions and others are "yes," GraphQL might be a good fit for what one is doing. I've personally had good experiences with it. It does bring in additional complexity. Is the complexity worth taking on in a specific dev team? This is something that will depend on the team. It is usually the case that a tool is not useful outside of the set of challenges it is tailored to address. And in some cases a tool is not well designed or implemented. It's always good to use the right tool for the job, at the level of complexity that a team is in a good position to handle.
- gsvclass 6y ago
  You can use GraphJin it automatically compiles GraphQL queries into Postgres SQL ones. The core idea is to simplify frontend development https://github.com/dosco/graphjin https://github.com/dosco/graphjin
- collyw 6y ago
  I keep hearing that you can specify the only fields that you need, but to be honest that isn't a problem we have. I have only taken a cursory glance, but adding a schema in at the request level, which then needs translated to a Mongo query just seems to be adding complexity rather than removing it. And URL's are going to be ugly if you are using GET requests, one of the things I am pushing for with a REST API. (Currently our API is not RESTful and most requests go over POSTs, giving less information when debugging).