The way I understand it, WAL-E is just something that you can put into pg's `archive_command` and `restore_command`, but what does it add other than that? Wher

The way I understand it, WAL-E is just something that you can put into pg's `archive_command` and `restore_command`, but what does it add other than that? Where I was, we investigated wal-e and determined it doesn't do anything that the raw curl commands wouldn't be able to do for us. What we decided to do was to utilize HDFS for continuous archival, and we patched a version of `pg_receivexlog` to actually stream the WAL files out to HDFS, with flushing support that acknowledged transactions over the wire when the flushes were complete. With this, you could treat this patched pg_receivexlog_hdfs command as a standby postgres database, and even add it to `synchronous_standby_names`, and postgres would effectively have its transaction logs synchronously written out to durable storage. We combined it with a lot of plumbing and we were able to get postgres running in mesos with docker without any actual persistent storage volumes (just used local disk.) The best part is, you'd do a COMMIT and it would essentially block until the data was in HDFS. No periodic snapshots where you'd lose transactions that happened seconds after the snapshot... if the client sees a transaction as committed, it's on durable storage. Worked pretty well, I'm surprised WAL-E doesn't support something similar (it only checkpoints at predefined intervals, not on transaction commit time.)