3 ms·

I would pay for this. The product you should be building is replication (what you have actually built) and wal-e configured to backup to amazon s3 (where I pro

I would pay for this. The product you should be building is replication (what you have actually built) and wal-e configured to backup to amazon s3 (where I provide the bucket). you guarantee that the backups are happening, build in some intelligence to make sure catastrophic "DELETE" statements would trigger a backup first, etc. Give me the ability to spawn a replica using the exact backup that I choose, etc. This is something I could totally pay for!

craigkerstiens 10y ago
I can definitely see the demand for something like this, but the engineering on it becomes quite difficult. Without the ability to cancel long running queries there can be some replication lag and the WAL disk could fill up. Without actual administrative access to the instance it becomes very hard to fully guarantee such replication. If you do want this more out of the box the real options today are either built it yourself or choose a provider that's delivering it.
- EmielMols 10y ago
  I don't think this is a huge problem. Yes, long running queries might introduce lag. When this lag hits a threshold of your choosing, PGBackup (or any service that would offer it) would then gladly send out pager alerts. The same would happen if the WAL could not be retrieved in time (because wal storage full): PGBackup would start a new base_backup and send out pager alerts.
EmielMols 10y ago
Thanks for your feedback! Are you comfortable with replicating your unencrypted data for your use case? The base_backup+xlog backup is going to be our next feature. Backing those up to s3: possibly. Encrypted?
- sandGorgon 10y ago
  good point. make your tech so that you can guarantee that you will never have access to customers data. If there is a replication failure.. so you kill the machine and respawn it kind of a thing. where would you backup if not s3 ? your own ? that would be a BIG oops...

craigkerstiens 10y ago

I can definitely see the demand for something like this, but the engineering on it becomes quite difficult. Without the ability to cancel long running queries there can be some replication lag and the WAL disk could fill up. Without actual administrative access to the instance it becomes very hard to fully guarantee such replication. If you do want this more out of the box the real options today are either built it yourself or choose a provider that's delivering it.

EmielMols 10y ago
I don't think this is a huge problem. Yes, long running queries might introduce lag. When this lag hits a threshold of your choosing, PGBackup (or any service that would offer it) would then gladly send out pager alerts. The same would happen if the WAL could not be retrieved in time (because wal storage full): PGBackup would start a new base_backup and send out pager alerts.

EmielMols 10y ago

Thanks for your feedback! Are you comfortable with replicating your unencrypted data for your use case? The base_backup+xlog backup is going to be our next feature. Backing those up to s3: possibly. Encrypted?

sandGorgon 10y ago
good point. make your tech so that you can guarantee that you will never have access to customers data. If there is a replication failure.. so you kill the machine and respawn it kind of a thing. where would you backup if not s3 ? your own ? that would be a BIG oops...