5 ms·

Running PG bouncer is a very basic optimization. You typically start out with PG bouncer in your stack if you have experience running Postgres. If you're new t

Running PG bouncer is a very basic optimization. You typically start out with PG bouncer in your stack if you have experience running Postgres. If you're new to running your own postgres databases you should also check out Wall-e: https://github.com/wal-e/wal-e https://github.com/wal-e/wal-e And the awesome pg stat statements https://www.postgresql.org/docs/10/static/pgstatstatements.html https://www.postgresql.org/docs/10/static/pgstatstatements.h...

derefr 9y ago
I’ve set up both pgbouncer and wal-e in the past, and they’re both worth it to have from effectively the very beginning. But it leaves me wondering: is there any sort of packaging of Postgres and these “friends” into a single opinionated virtual appliance, such that I could just stick up a couple of VMs with the same image with different role tags, and get a good cluster with automatic transparent N:M proxying, automatic transparent backup-and-restore, etc? In other words, is there a product that is to Postgres as Gitlab CE is to Github (or like Dokku is to Heroku): a “host it yourself, but easily, and starting from a scale of 1 with no overhead” equivalent of a SaaS service?
- snuxoll 9y ago
  If you're deploying on K8S at least Crunchy Data has you covered [0], I haven't used it myself but it's the closest I've been able to find without going with something like EnterpriseDB. I'm pretty sure there's some ansible/puppet/chef/saltstack code out there to build something similar as well. [0] https://github.com/CrunchyData/crunchy-containers https://github.com/CrunchyData/crunchy-containers
  - mslot 9y ago
    There's a postgres operator by Zalando which sets up postgres with WAL-E and auto failover using Patroni. https://github.com/zalando-incubator/postgres-operator https://github.com/zalando-incubator/postgres-operator
craigkerstiens 9y ago
If you are already familiar with wal-e, or even if not, you might want to consider taking a look at wal-g[1]. Wal-g is a newer edition of wal-e written in go that we've seen can have up to 7x performance improvements[2]. [1] https://github.com/wal-g/wal-g https://github.com/wal-g/wal-g [2] https://www.citusdata.com/blog/2017/08/18/introducing-wal-g-faster-restores-for-postgres/ https://www.citusdata.com/blog/2017/08/18/introducing-wal-g-...
- djd20 9y ago
  While wal-g looks great, having evaluated it a bit, I worry that nobody seems to be maintaining it based on issues/pr's I looked at.
  - djd20 9y ago
    Scratch that - I see fresh commits - will give it another spin!
- briffle 9y ago
  I really wanted to use wal-g, but they had NO support for any backend storage besides S3, and did not appear to have any interest in developing more backends.. (We use GCP)

derefr 9y ago

I’ve set up both pgbouncer and wal-e in the past, and they’re both worth it to have from effectively the very beginning. But it leaves me wondering: is there any sort of packaging of Postgres and these “friends” into a single opinionated virtual appliance, such that I could just stick up a couple of VMs with the same image with different role tags, and get a good cluster with automatic transparent N:M proxying, automatic transparent backup-and-restore, etc? In other words, is there a product that is to Postgres as Gitlab CE is to Github (or like Dokku is to Heroku): a “host it yourself, but easily, and starting from a scale of 1 with no overhead” equivalent of a SaaS service?

snuxoll 9y ago
If you're deploying on K8S at least Crunchy Data has you covered [0], I haven't used it myself but it's the closest I've been able to find without going with something like EnterpriseDB. I'm pretty sure there's some ansible/puppet/chef/saltstack code out there to build something similar as well. [0] https://github.com/CrunchyData/crunchy-containers https://github.com/CrunchyData/crunchy-containers
- mslot 9y ago
  There's a postgres operator by Zalando which sets up postgres with WAL-E and auto failover using Patroni. https://github.com/zalando-incubator/postgres-operator https://github.com/zalando-incubator/postgres-operator

craigkerstiens 9y ago

If you are already familiar with wal-e, or even if not, you might want to consider taking a look at wal-g[1]. Wal-g is a newer edition of wal-e written in go that we've seen can have up to 7x performance improvements[2]. [1] https://github.com/wal-g/wal-g https://github.com/wal-g/wal-g [2] https://www.citusdata.com/blog/2017/08/18/introducing-wal-g-faster-restores-for-postgres/ https://www.citusdata.com/blog/2017/08/18/introducing-wal-g-...

djd20 9y ago
While wal-g looks great, having evaluated it a bit, I worry that nobody seems to be maintaining it based on issues/pr's I looked at.
- djd20 9y ago
  Scratch that - I see fresh commits - will give it another spin!
briffle 9y ago
I really wanted to use wal-g, but they had NO support for any backend storage besides S3, and did not appear to have any interest in developing more backends.. (We use GCP)