4 ms·

does anyone know how this compares to wal-e ?

brightball 10y ago
Was about to ask the same question. I haven't heard of Barman 1.0 before.
tzaman 10y ago
I don't have any experience with wal-e, but based on what I learned during my research (went with barman), there is no need to to install barman on the primary or the standby server, just a separate instance with proper user/permissions. This separate instance is then called from primary's "archive_command", which usually looks like: archive_command = 'rsync -a %p barman@10.0.0.1:/var/lib/barman/main-db-server/incoming/%f' and once rsync'd barman does it's own magic on it (thru cron). Based on my observations, wal-e is more powerful if you want to store archives remotely (and is primarily an archiving solution), and barman is kinda the official backup solution (2ndQuadrant is a big contributor in the community) plus it offers point-in-time recovery. Barman + Repmgr (which can do an automatic failover or a manual switchover) is a powerful combo. If only there were reliable tools to determine the master and slave (like Redis Sentinels)
- amenonsen 10y ago
  Recent versions of Barman allow you to do WAL archiving without using an rsync/ssh-based archive_command. Instead, the backup server uses streaming replication via the standard pg_receivexlog command to build up a WAL archive. The big new feature in Barman 2.0 is that it can take base backups using pg_basebackup, rather than rsync. This means that it can be used without SSH access between servers (only replication connections to Postgres on port 5432).
amenonsen 10y ago
The big conceptual difference is that with WAL-E, all your backup code and configuration is on your Postgres server, whereas with Barman, it's all (or almost all) on your backup server. WAL-E stores backups in S3 (or S3-alikes), whereas Barman is built around the idea of having a backup server (it can store backups in S3 too, but it's not central to its operation as it is with WAL-E).
- sandGorgon 10y ago
  hey - thanks for this. it makes things much simpler. This then begs the question - can this run on AWS Lambda and archive to s3 ? Because wal-e + s3 is killer. I seriously dont think that the headache of managing a backup server is worth it. Is there is a situation where barman's usecase is superior to wal-e ?
  - gbartolini 10y ago
    Some food for thought. These are the first cases that came to my mind: 1) Zero data loss backups (as far as I know, Barman is the first backup tool for PostgreSQL to implement it) 2) Monitoring ('check' command) 3) Hybrid WAL shipping (archiving + streaming) 4) Incremental backups 5) Faster recovery time (backups are stored as plain files, you can have incremental recovery and you can use 'get-wal') - also we are working on parallel backup and recovery 6) Backup from a standby In terms of management, Barman is very low maintenance, but the outcomes you get are relevant. If you have multiple PostgreSQL servers, you can concentrate backups in a single Barman host. Every server can have its own settings allowing you to implement hybrid solutions (you can mix versions, backup methods such as rsync or pg_basebackup, WAL archiving, WAL streaming, and so on). As said above by "amenonsen", with hook scripts you can also relay WAL files to S3 very easily, and create a backup of your backup. That is why I think having a separate backup server has more pros than cons. I hope this helps.
    - sandGorgon 10y ago
      thanks - this was very useful!

Was about to ask the same question. I haven't heard of Barman 1.0 before.

I don't have any experience with wal-e, but based on what I learned during my research (went with barman), there is no need to to install barman on the primary or the standby server, just a separate instance with proper user/permissions. This separate instance is then called from primary's "archive_command", which usually looks like: archive_command = 'rsync -a %p barman@10.0.0.1:/var/lib/barman/main-db-server/incoming/%f' and once rsync'd barman does it's own magic on it (thru cron). Based on my observations, wal-e is more powerful if you want to store archives remotely (and is primarily an archiving solution), and barman is kinda the official backup solution (2ndQuadrant is a big contributor in the community) plus it offers point-in-time recovery. Barman + Repmgr (which can do an automatic failover or a manual switchover) is a powerful combo. If only there were reliable tools to determine the master and slave (like Redis Sentinels)

amenonsen 10y ago
Recent versions of Barman allow you to do WAL archiving without using an rsync/ssh-based archive_command. Instead, the backup server uses streaming replication via the standard pg_receivexlog command to build up a WAL archive. The big new feature in Barman 2.0 is that it can take base backups using pg_basebackup, rather than rsync. This means that it can be used without SSH access between servers (only replication connections to Postgres on port 5432).

amenonsen 10y ago

The big conceptual difference is that with WAL-E, all your backup code and configuration is on your Postgres server, whereas with Barman, it's all (or almost all) on your backup server. WAL-E stores backups in S3 (or S3-alikes), whereas Barman is built around the idea of having a backup server (it can store backups in S3 too, but it's not central to its operation as it is with WAL-E).

sandGorgon 10y ago
hey - thanks for this. it makes things much simpler. This then begs the question - can this run on AWS Lambda and archive to s3 ? Because wal-e + s3 is killer. I seriously dont think that the headache of managing a backup server is worth it. Is there is a situation where barman's usecase is superior to wal-e ?
- gbartolini 10y ago
  Some food for thought. These are the first cases that came to my mind: 1) Zero data loss backups (as far as I know, Barman is the first backup tool for PostgreSQL to implement it) 2) Monitoring ('check' command) 3) Hybrid WAL shipping (archiving + streaming) 4) Incremental backups 5) Faster recovery time (backups are stored as plain files, you can have incremental recovery and you can use 'get-wal') - also we are working on parallel backup and recovery 6) Backup from a standby In terms of management, Barman is very low maintenance, but the outcomes you get are relevant. If you have multiple PostgreSQL servers, you can concentrate backups in a single Barman host. Every server can have its own settings allowing you to implement hybrid solutions (you can mix versions, backup methods such as rsync or pg_basebackup, WAL archiving, WAL streaming, and so on). As said above by "amenonsen", with hook scripts you can also relay WAL files to S3 very easily, and create a backup of your backup. That is why I think having a separate backup server has more pros than cons. I hope this helps.
  - sandGorgon 10y ago
    thanks - this was very useful!