5 ms·

I’m interested in an easy backup-out-of-aws solution. We use snapshots for regular backups, and have implemented a solutions to share and copy snapshots to a d

by aquark 7y ago

I’m interested in an easy backup-out-of-aws solution. We use snapshots for regular backups, and have implemented a solutions to share and copy snapshots to a different aws account. But given that our production database is absolutely vital to the existence of the company I’d like to have an easy way to create a secure ‘not affliated with Amazon’ copy regularly in case of the nightmare scenario where an account is compromised/terminated

rsync 7y ago
"I’m interested in an easy backup-out-of-aws solution." rsync.net has 'rclone' built into the environment[1] so it is trivially easy to call rclone, over SSH: ssh user@rsync.net rclone s3:/some/bucket /rsync/account ... and perform arbitrary data movements into, out of, and between S3/glacier/nearline/gdrive resources. Your "out-of-aws solution" contains ZFS snapshots, on any schedule you like, so you can just do a "dumb" sync to rsync.net and handle the retention with the snapshot schedule. Also, the snapshots are immutable (read-only) so your data gains some protection against ransomware and other threats like accidental deletion. You should trust us with your backups because we've been doing this since 2001. [1] https://rsync.net/products/rclone.html https://rsync.net/products/rclone.html
- aquark 7y ago
  Great, thanks -- will take a look at this. The challenge with RDS in the past has always been getting the data easily and scriptable into an S3/etc bucket to start with. This might be a good enough solution. Restore would be a pain, but at the point its needed the pain would be acceptable.
  - rsync 7y ago
    "Restore would be a pain, but at the point its needed the pain would be acceptable." I can't speak specifically to this, but I will point out that you can just as easily run those 'rclone' commands in reverse, to move data from an rsync.net snapshot out to (insert cloud): ssh user@rsync.net rclone .zfs/snapshot/1dayago/bucket s3:/some/bucket
    - sandGorgon 7y ago
      AFAIK rsync.net doesnt work with RDS (which is a managed solution with no ssh access).
      - rsync 7y ago
        I can't speak to RDS specifically, but if you can get RDS into either S3 or Glacier, then there is an easy path to rsync.net: rclone. The ssh access is not to Amazon, or S3 or RDS, etc. - you run the rclone command at rsync.net, over ssh: ssh user@rsync.net rclone s3:/some/bucket local/rsync.net/dir
Alex3917 7y ago
> We use snapshots for regular backups, and have implemented a solutions to share and copy snapshots to a different aws account. How do you do this if the db is encrypted? Or rather, how do you restore from the encrypted backup in the second account if the master key in the original account is somehow deleted without anyone noticing and intervening before the deletion becomes finalized?
- aquark 7y ago
  Good point ... we don't have this db encrypted as the DBMS level. Internally sensitive information is encrypted using keys not managed by KMS

by aquark 7y ago

rsync 7y ago

"I’m interested in an easy backup-out-of-aws solution." rsync.net has 'rclone' built into the environment[1] so it is trivially easy to call rclone, over SSH: ssh user@rsync.net rclone s3:/some/bucket /rsync/account ... and perform arbitrary data movements into, out of, and between S3/glacier/nearline/gdrive resources. Your "out-of-aws solution" contains ZFS snapshots, on any schedule you like, so you can just do a "dumb" sync to rsync.net and handle the retention with the snapshot schedule. Also, the snapshots are immutable (read-only) so your data gains some protection against ransomware and other threats like accidental deletion. You should trust us with your backups because we've been doing this since 2001. [1] https://rsync.net/products/rclone.html https://rsync.net/products/rclone.html

aquark 7y ago
Great, thanks -- will take a look at this. The challenge with RDS in the past has always been getting the data easily and scriptable into an S3/etc bucket to start with. This might be a good enough solution. Restore would be a pain, but at the point its needed the pain would be acceptable.
- rsync 7y ago
  "Restore would be a pain, but at the point its needed the pain would be acceptable." I can't speak specifically to this, but I will point out that you can just as easily run those 'rclone' commands in reverse, to move data from an rsync.net snapshot out to (insert cloud): ssh user@rsync.net rclone .zfs/snapshot/1dayago/bucket s3:/some/bucket
  - sandGorgon 7y ago
    AFAIK rsync.net doesnt work with RDS (which is a managed solution with no ssh access).
    - rsync 7y ago
      I can't speak to RDS specifically, but if you can get RDS into either S3 or Glacier, then there is an easy path to rsync.net: rclone. The ssh access is not to Amazon, or S3 or RDS, etc. - you run the rclone command at rsync.net, over ssh: ssh user@rsync.net rclone s3:/some/bucket local/rsync.net/dir

Alex3917 7y ago

> We use snapshots for regular backups, and have implemented a solutions to share and copy snapshots to a different aws account. How do you do this if the db is encrypted? Or rather, how do you restore from the encrypted backup in the second account if the master key in the original account is somehow deleted without anyone noticing and intervening before the deletion becomes finalized?

aquark 7y ago
Good point ... we don't have this db encrypted as the DBMS level. Internally sensitive information is encrypted using keys not managed by KMS