3 ms·

Very few organizations in the world truly need to scale out their database. Their problem is usually that of a bad design. For those that need to scale out,

Very few organizations in the world truly need to scale out their database. Their problem is usually that of a bad design. For those that need to scale out, NoSQL is not a magic solution either. The scaling out solution pretty much these days for SQL database is replication, sharding and partitioning tables. Postgres does these very well.

threeseed 10y ago
You might want to learn a bit about "big data". Almost every major organisation will have a big data analytics program doing feature extraction, modelling, machine learning etc. Usually this starts with Hadoop/Spark with data in HDFS but then a point comes when you want this in a database. PostgreSQL is awful at this which is why almost no uses it in this space. Apart from the lack of native drivers it is simply too difficult to do basic clustering. And the fact that it isn't built into the product doesn't give you a lot of confidence that it is (a) going to work and (b) is going to be supported. And yes for those that scale out NoSQL is a magic solution. That's why they are so popular. I can deploy a 40 node Cassandra cluster in 30 mins and guarantee it works. Likewise MongoDB replica sets are ridiculously easy.
- collyw 10y ago
  Maybe you could just design your data a bit better with smaller data types, and keep it all one one machine.
forgotpwtomain 10y ago
> The scaling out solution pretty much these days for SQL database is replication, sharding and partitioning tables. Postgres does these very well. I tend to disagree that Postgres does partitioning well; partioning table resolution is rather dumb, there is no hinting (e.g. there is no way to tell postgres that there is a unique mapping for each query to a table) and no caching of CHECK results.
gnaritas 10y ago
> Very few organizations in the world truly need to scale out their database. Everyone organization that has a database wants to scale out to at least a 2 node cluster just for not having a single point of failure. Scaling is not always about performance, at first, it's more about availability.

threeseed 10y ago

You might want to learn a bit about "big data". Almost every major organisation will have a big data analytics program doing feature extraction, modelling, machine learning etc. Usually this starts with Hadoop/Spark with data in HDFS but then a point comes when you want this in a database. PostgreSQL is awful at this which is why almost no uses it in this space. Apart from the lack of native drivers it is simply too difficult to do basic clustering. And the fact that it isn't built into the product doesn't give you a lot of confidence that it is (a) going to work and (b) is going to be supported. And yes for those that scale out NoSQL is a magic solution. That's why they are so popular. I can deploy a 40 node Cassandra cluster in 30 mins and guarantee it works. Likewise MongoDB replica sets are ridiculously easy.

collyw 10y ago
Maybe you could just design your data a bit better with smaller data types, and keep it all one one machine.

forgotpwtomain 10y ago

> The scaling out solution pretty much these days for SQL database is replication, sharding and partitioning tables. Postgres does these very well. I tend to disagree that Postgres does partitioning well; partioning table resolution is rather dumb, there is no hinting (e.g. there is no way to tell postgres that there is a unique mapping for each query to a table) and no caching of CHECK results.

gnaritas 10y ago

> Very few organizations in the world truly need to scale out their database. Everyone organization that has a database wants to scale out to at least a 2 node cluster just for not having a single point of failure. Scaling is not always about performance, at first, it's more about availability.