3 ms·

Considering their "Continuous Protection" feature, could they be streaming a write log out to S3? I think that way they could use the backup as the sole guarant

Considering their "Continuous Protection" feature, could they be streaming a write log out to S3? I think that way they could use the backup as the sole guarantee of durability, and use instance storage for the database. EDIT: also, I'm not sure if this is generally known elsewhere, but might they not be on ec2, they are advertising 1.7 GB cache, which is the same as a small instance, except without any room for the operating system or postgres itself.

imbriaco 15y ago
You're exactly right about log streaming to S3. We open sourced the tool that we use to perform that: https://github.com/heroku/wal-e https://github.com/heroku/wal-e
- bdonlan 15y ago
  S3 doesn't support continual logging - you need to decide on a file size up-front. Are you creating a new object for each log entry, or is there a window in which commits can be lost before the log gets flushed out, or do you have some other workaround...?
  - joevandyk 15y ago
    There's a window of a couple minutes in which data could be lost (in the worst case scenario).
    - sector 15y ago
      So it's postgres log shipping, with S3 as the target?
      - deleted 15y ago
        [deleted]
      - joevandyk 15y ago
        Yup.
- jaylevitt 15y ago
  So would I need to (learn and) use PG9's replication with this? Or is one ephemeral-stored database plus shipped logs sufficient, assuming I can live with some downtime if the instance disappears? As an experienced techie but a PG novice, what do I need to go learn about PG recovery procedures to safely use this?