5 ms·

What do you mean pG isn't designed for scalability? Its SMP scaling is incredibly good, close to linear.

by hoodoof 10y ago

manigandham 10y ago
Postgres neither scales vertically or horizontally. v9.6 finally brought some parallel scans but is still limited in scope and far behind the commercial databases. Same with replication and failover, although there are decent 3rd party extensions to get it working.
- anarazel 10y ago
  I suppose the parent comment was more about scaling up for transactional workloads - the story there is a lot better (although there's still issues left we haven't tackled, but mostly on very big machines) than for analytics workloads. But yes, while we progressed in 9.6, there's still a lot of important things lacking to scale a larger fraction of analytics queries.
- flukus 10y ago
  The vast majority of us probably won't ever need that scalability anyway. Most places I've seen that needed something more scalable needed to just stop being so retarded with the tools they already had.
  - drdaeman 10y ago
    As for the clustering in general... Scalability may be not important, but replication and failover should be. I believe, every place that had existed long enough must've had theirs "ouch, our master DB host went down" moment. I wasn't able to set it up properly some years ago and settled down with warm standby replica (streaming + WAL shipping) and manual failover. Automating it (and even automating the recovery when old master is back online and healthy) was certainly possible - and I got the overall idea how it should operate. But the effort required to set it up was just too big for me, so I decided it wasn't worth the hassle and settled down with "uh, if it dies, we'll get alerted and switch to the backup server manually" scenario. Having something that goes in line of "you start a fresh server, tell it some other server address to peer with, and it does the magic (with exact guarantees and drawbacks noted in documentation)" would be really awesome. RethinkDB is just like that. PostgreSQL - at least 8.4 - wasn't, unless I've really missed something. I haven't yet checked newer versions' features in any detail, so not sure about 8.5/8.6.
    - takeda 10y ago
      You can try http://repmgr.org/ http://repmgr.org/ for automation. It offers easy commands to perform failover and even has an option to configure automatic one. After reading about github issues[1], I am a bit cautious about having automatic failover though. [1] https://github.com/blog/1261-github-availability-this-week https://github.com/blog/1261-github-availability-this-week
      - drdaeman 10y ago
        Thanks for the link, I think I haven't saw this one before. As for automation... Things can always go wrong, sure. But I wonder how many times HA and automatic failover had saved the day at GitHub so no outside observers had a faintest idea there was something failing in there.
    - manigandham 10y ago
      > you start a fresh server, tell it some other server address to peer with, and it does the magic This is the one thing I ask of every database software, yet we still don't really have it. 90% of problems could be solved if there was a focus on the basics like easy startup, config and clustering.
      - flukus 10y ago
        SQL Server is almost that easy, but still doesn't handle schema changes well.
        manigandham 10y ago
        How so? Aerospike, ScyllaDB, Rethink, MemSQL, Redis are the only databases that get close to this. SQL Server availability groups requires Windows Server Failover Clustering, which is not quick or easy.
  - manigandham 10y ago
    Vertical scalability is important for everyone as it makes better use of hardware (lower costs or more performance). Horizontal scalability is important for HA which every production environment would like or need.

What do you mean pG isn't designed for scalability? Its SMP scaling is incredibly good, close to linear.

by hoodoof 10y ago

What do you mean pG isn't designed for scalability? Its SMP scaling is incredibly good, close to linear.

Postgres neither scales vertically or horizontally. v9.6 finally brought some parallel scans but is still limited in scope and far behind the commercial databases. Same with replication and failover, although there are decent 3rd party extensions to get it working.

anarazel 10y ago
I suppose the parent comment was more about scaling up for transactional workloads - the story there is a lot better (although there's still issues left we haven't tackled, but mostly on very big machines) than for analytics workloads. But yes, while we progressed in 9.6, there's still a lot of important things lacking to scale a larger fraction of analytics queries.
flukus 10y ago
The vast majority of us probably won't ever need that scalability anyway. Most places I've seen that needed something more scalable needed to just stop being so retarded with the tools they already had.
- drdaeman 10y ago
  As for the clustering in general... Scalability may be not important, but replication and failover should be. I believe, every place that had existed long enough must've had theirs "ouch, our master DB host went down" moment. I wasn't able to set it up properly some years ago and settled down with warm standby replica (streaming + WAL shipping) and manual failover. Automating it (and even automating the recovery when old master is back online and healthy) was certainly possible - and I got the overall idea how it should operate. But the effort required to set it up was just too big for me, so I decided it wasn't worth the hassle and settled down with "uh, if it dies, we'll get alerted and switch to the backup server manually" scenario. Having something that goes in line of "you start a fresh server, tell it some other server address to peer with, and it does the magic (with exact guarantees and drawbacks noted in documentation)" would be really awesome. RethinkDB is just like that. PostgreSQL - at least 8.4 - wasn't, unless I've really missed something. I haven't yet checked newer versions' features in any detail, so not sure about 8.5/8.6.
  - takeda 10y ago
    You can try http://repmgr.org/ http://repmgr.org/ for automation. It offers easy commands to perform failover and even has an option to configure automatic one. After reading about github issues[1], I am a bit cautious about having automatic failover though. [1] https://github.com/blog/1261-github-availability-this-week https://github.com/blog/1261-github-availability-this-week
    - drdaeman 10y ago
      Thanks for the link, I think I haven't saw this one before. As for automation... Things can always go wrong, sure. But I wonder how many times HA and automatic failover had saved the day at GitHub so no outside observers had a faintest idea there was something failing in there.
  - manigandham 10y ago
    > you start a fresh server, tell it some other server address to peer with, and it does the magic This is the one thing I ask of every database software, yet we still don't really have it. 90% of problems could be solved if there was a focus on the basics like easy startup, config and clustering.
    - flukus 10y ago
      SQL Server is almost that easy, but still doesn't handle schema changes well.
      - manigandham 10y ago
        How so? Aerospike, ScyllaDB, Rethink, MemSQL, Redis are the only databases that get close to this. SQL Server availability groups requires Windows Server Failover Clustering, which is not quick or easy.
- manigandham 10y ago
  Vertical scalability is important for everyone as it makes better use of hardware (lower costs or more performance). Horizontal scalability is important for HA which every production environment would like or need.