5 ms·

Is anyone using either S3 or Glacier to store encrypted backups of their personal computer(s)? I've only used Time Machine to back up my machine for a long time

Is anyone using either S3 or Glacier to store encrypted backups of their personal computer(s)? I've only used Time Machine to back up my machine for a long time, but I don't really trust it and would like to have another back up in the cloud. Any tools that automate back up and restore to/from S3/Glacier? What are your experiences?

res0nat0r 10y ago
Check out Duplicity or Duplicati. Also Arq for OSX I believe is very nice and has a great GUI but isn't free.
- jonatron 10y ago
  +1 for Duplicity
icecreammatt 10y ago
I use Arq (https://www.arqbackup.com/ https://www.arqbackup.com/) and it works very well. I've only tested retrieving small amounts of data from it so I can't comment much on a large bill. I only wish it worked on Linux. I've been thinking about seeing if it would work with Wine.
- chrislund 10y ago
  Ditto. I've been using Arq for 4 years now to back up nearly a TB of data to S3 and Google Cloud Storage.
  - CoachRufus87 10y ago
    Out of curiosity, why both?
    - chrislund 10y ago
      I guess mostly to not have all my cloud backup eggs in one basket.
- astrostl 10y ago
  Also recommend Arq against any service. At a certain scale, especially after factoring bandwidth for restores, Amazon Cloud Drive at a flat $60/year becomes more attractive.
- brandon272 10y ago
  +1 for Arq!
fmihaila 10y ago
I use Tarsnap (https://www.tarsnap.com/ https://www.tarsnap.com/), which uses S3. It's true, though, that I only backup a small enough subset of my files, not anywhere close to a complete image. OTOH, it's very easy to use (CLI only), and as secure as it gets.
jeromegv 10y ago
Arq on OSX is pretty great.
- x0x0 10y ago
  I second this. It costs about $50 for a license but they support incremental backups up to any of a NAS, s3, s3 glacier, various tiers of google cloud storage, dropbox, and google drive. Client-side encryption is built in. It's good software.
c13k 10y ago
Using S4cmd on my Debian box I backup copies of my entire Lightroom folder structure contents. All my personal photos and videos get uploaded to a bucket on S3, then I convert the entire bucket to Glacier. Now with the new pricing of $0.004/GB and easy retrieval, it's a very nice setup. To backup nearly a terabyte of photos costs about $4/month in storage costs. Uploading costs a bit extra due to the pricing for requests. https://github.com/bloomreach/s4cmd https://github.com/bloomreach/s4cmd
- GordonS 10y ago
  How much would it cost if you ever needed to restore that TB? I looked into Glacier a couple of years ago, and, from memory, restore costs were insane.
boulos 10y ago
I use my Synology NAS pointed at Google Nearline (through the included S3 support!). GCS also supports customer supplied encryption keys, but I store the bytes as encrypted on the NAS box itself. As others have said, if you're just trying to back up your Mac, take a look at Arq. Disclosure: I work on Google Cloud (and get $30/month of credit, so my first 3 TB are free).
tombrossman 10y ago
I have about 150GB in /home backed up daily using Duplicity, which encrypts & compresses everything and saves to a second internal backup drive. Data is kept for 6 months minimum. After several years, the total backup size is 190GB which syncs daily to an S3 bucket and my monthly bills are about $11USD. If I ever had to restore all that from S3 it would cost extra but would not be prohibitively expensive. Install the AWS CLI (https://aws.amazon.com/cli/ https://aws.amazon.com/cli/) and choose whatever method you like for making an encrypted local backup. Then sync that backup partition to S3 every day. Here's an example command you can adapt for cron to call via a shell script: /home/tom/.local/bin/aws s3 sync /mnt/backups/daily s3://your-s3-bucket-name --storage-class STANDARD_IA --acl private --sse The -sse means server side encryption which is redundant since I've encrypted the data prior to uploading, but why not?
ekimekim 10y ago
I was a masochist and hand-rolled incremental tar snapshots to Glacier using a cronjob. It works great for keeping price down, though I learnt the "lots of retrievals" lesson the hard way once I actually tried to retrieve my data. I now compromise by doing a full snapshot monthly and incremental daily, so I'm upper bound to 30 retrievals. I'd highly recommend not repeating my mistake - use a real backup service for your actual data. Though rolling your own can be fun and interesting, it's probably a bad idea to bet your data on it.
estefan 10y ago
Check out rclone [1]. I haven't used it yet but it's free and supports lots of backends. [1] http://rclone.org/ http://rclone.org/
Twirrim 10y ago
I use duplicity to backup my VPS to S3, and Fast Glacier to archive stuff to Glacier from my Windows machine (things like photographs, important documents etc.) Note: Glacier is not a backup service, it's an archive service. It's for long term backups, vs the relatively short term of standard backups (Full, delta, delta.. cycles). With Glacier if you delete an archive before 90 days, you'll still be charged for the full 90 days of storage.

res0nat0r 10y ago

Check out Duplicity or Duplicati. Also Arq for OSX I believe is very nice and has a great GUI but isn't free.

jonatron 10y ago
+1 for Duplicity

icecreammatt 10y ago

I use Arq (https://www.arqbackup.com/ https://www.arqbackup.com/) and it works very well. I've only tested retrieving small amounts of data from it so I can't comment much on a large bill. I only wish it worked on Linux. I've been thinking about seeing if it would work with Wine.

chrislund 10y ago
Ditto. I've been using Arq for 4 years now to back up nearly a TB of data to S3 and Google Cloud Storage.
- CoachRufus87 10y ago
  Out of curiosity, why both?
  - chrislund 10y ago
    I guess mostly to not have all my cloud backup eggs in one basket.
astrostl 10y ago
Also recommend Arq against any service. At a certain scale, especially after factoring bandwidth for restores, Amazon Cloud Drive at a flat $60/year becomes more attractive.
brandon272 10y ago
+1 for Arq!

fmihaila 10y ago

I use Tarsnap (https://www.tarsnap.com/ https://www.tarsnap.com/), which uses S3. It's true, though, that I only backup a small enough subset of my files, not anywhere close to a complete image. OTOH, it's very easy to use (CLI only), and as secure as it gets.

jeromegv 10y ago

Arq on OSX is pretty great.

x0x0 10y ago
I second this. It costs about $50 for a license but they support incremental backups up to any of a NAS, s3, s3 glacier, various tiers of google cloud storage, dropbox, and google drive. Client-side encryption is built in. It's good software.

c13k 10y ago

Using S4cmd on my Debian box I backup copies of my entire Lightroom folder structure contents. All my personal photos and videos get uploaded to a bucket on S3, then I convert the entire bucket to Glacier. Now with the new pricing of $0.004/GB and easy retrieval, it's a very nice setup. To backup nearly a terabyte of photos costs about $4/month in storage costs. Uploading costs a bit extra due to the pricing for requests. https://github.com/bloomreach/s4cmd https://github.com/bloomreach/s4cmd

GordonS 10y ago
How much would it cost if you ever needed to restore that TB? I looked into Glacier a couple of years ago, and, from memory, restore costs were insane.

boulos 10y ago

I use my Synology NAS pointed at Google Nearline (through the included S3 support!). GCS also supports customer supplied encryption keys, but I store the bytes as encrypted on the NAS box itself. As others have said, if you're just trying to back up your Mac, take a look at Arq. Disclosure: I work on Google Cloud (and get $30/month of credit, so my first 3 TB are free).

tombrossman 10y ago

I have about 150GB in /home backed up daily using Duplicity, which encrypts & compresses everything and saves to a second internal backup drive. Data is kept for 6 months minimum. After several years, the total backup size is 190GB which syncs daily to an S3 bucket and my monthly bills are about $11USD. If I ever had to restore all that from S3 it would cost extra but would not be prohibitively expensive. Install the AWS CLI (https://aws.amazon.com/cli/ https://aws.amazon.com/cli/) and choose whatever method you like for making an encrypted local backup. Then sync that backup partition to S3 every day. Here's an example command you can adapt for cron to call via a shell script: /home/tom/.local/bin/aws s3 sync /mnt/backups/daily s3://your-s3-bucket-name --storage-class STANDARD_IA --acl private --sse The -sse means server side encryption which is redundant since I've encrypted the data prior to uploading, but why not?

ekimekim 10y ago

I was a masochist and hand-rolled incremental tar snapshots to Glacier using a cronjob. It works great for keeping price down, though I learnt the "lots of retrievals" lesson the hard way once I actually tried to retrieve my data. I now compromise by doing a full snapshot monthly and incremental daily, so I'm upper bound to 30 retrievals. I'd highly recommend not repeating my mistake - use a real backup service for your actual data. Though rolling your own can be fun and interesting, it's probably a bad idea to bet your data on it.

estefan 10y ago

Check out rclone [1]. I haven't used it yet but it's free and supports lots of backends. [1] http://rclone.org/ http://rclone.org/

Twirrim 10y ago

I use duplicity to backup my VPS to S3, and Fast Glacier to archive stuff to Glacier from my Windows machine (things like photographs, important documents etc.) Note: Glacier is not a backup service, it's an archive service. It's for long term backups, vs the relatively short term of standard backups (Full, delta, delta.. cycles). With Glacier if you delete an archive before 90 days, you'll still be charged for the full 90 days of storage.