4 ms·
This is great. Can't wait to be using it. We've been using WAL-E for years and this looks like a big improvement. The steady, high throughput is a big deal – o
by drob 9y ago
This is great. Can't wait to be using it.
We've been using WAL-E for years and this looks like a big improvement. The steady, high throughput is a big deal – our prod base backups take 36 hours to restore, so if the recovery speed improvements are as advertised, that's a big win. In the kind of situation in which we'd be using these, the difference between 9 hours and 36 hours is major.
Also, the quality of life improvements are great. Despite deploying WAL-E for years, we _still_ have problems with python, pip, dependencies, etc, so the switch to go is a welcome one. The backup_label issue has bitten us a half dozen times, and every time it's very scary for whoever is on-call. (The right thing to do is to rm a file in the database's main folder, so it's appropriately terrifying.) So switching to the new non-exclusive backups will also be great.
We're on 9.5 at the moment but will be upgrading to 10 after it comes out. Looking forward to testing this out. Awesome work!
- humanfromearth 9y agoSame here, perf is obviously great, but no more crazy dependencies is just great! So much time I wasted trying to make it fit into a docker container.
- gardnr 9y agoCan you share your dockerfiles?
- ozgune 9y agoHey Dan, nice to hear from you! > our prod base backups take 36 hours to restore, so if the recovery speed improvements are as advertised, that's a big win. Yes, if you attach 16TB of storage to each instance, your back-up restores may take a while. :))
- paulsutter 9y agoIs it possible to run a continuous restore in parallel with normal operation so that there's a warm standby (almost) ready to go? Especially in another data center?
- billhathaway 9y agoWould standard postgres streaming replication do what you are looking for? https://wiki.postgresql.org/wiki/Streaming_Replication https://wiki.postgresql.org/wiki/Streaming_Replication We use a combination of streaming replication and wal-e backups. A separate machine performs multiple restores per hour and verifies restores work ok and that the data is recent.
- bpicolo 9y agoYeah, just have a read replica. Can promote it to master later