5 ms·

Ask HN: What is the current state of the art in BIG (>5TB) cloud backups?

I'm talking about greater than 5 TB in size. Rclone looks really good because I can just give it a bandwidth limit, point it at google drive and fire and forget. But I'm curious if that is the best way to do this? What does HN think?

slipheen 11mo ago
Like so many things, it depends- How quickly do you need to be able to restore? Is it commercial or homelab? The most cost-effective option by far would be to put a NAS device someplace offsite. You could use tailscale to connect to it remotely. After that, depending on your access patterns, either a glacier-style s3 service (aws or backblaze/etc), or a rented bare-metal server with big disks some place inexpensive.
- jacobwilliamroy 11mo ago
  This is for a "warm" backup as opposed to a hot backup so restoration timeline would be like seven days.
monerozcash 11mo ago
This needs more information. 5TB is far from "BIG" when it comes to cloud backups. You can probably get away with google drive+rclone+borg/restic/whatever, but it will be rather clunky. Backblaze might be a nicer backend to use. I use rsync.net with borg, but not sure about your budget. Their 1TB lifetime plan is very competitive though.
- KomoD 11mo ago
  > Their 1TB lifetime plan is very competitive though. I know rsync.net has been around for a long time but I'm always very skeptical of lifetime plans, you'd have to use it ~10 years for it to be worth it. (Comparing with Hetzner's 1TB storage box)
Sohcahtoa82 11mo ago
5 TB? I don't know how to count that low.
- jacobwilliamroy 11mo ago
  It's just bigger than any amount I've ever dealt with in my whole life (I'm 30) and also it's hard to find solutions to manage it without running into bandwidth caps and also things are running for a very long time on the client side: days or weeks.
- carstenhag 11mo ago
  Probably just above normal b2c backup services, and below company requirements
brudgers 11mo ago
Why are you choosing to use the cloud instead of spinning rust under your control? Or to put it another way, why is state of the art important?
- kjkjadksj 11mo ago
  Because you want an offsite backup of your local backups.
- jacobwilliamroy 11mo ago
  I need to know what people are doing recently because most of the documentation I'm finding online is from 3-5 years ago and I want the most up-to-date information.
  - monerozcash 11mo ago
    People are doing pretty much exactly what they were doing 3-5 years ago, software has been pretty good for a while no so there hasn't been much change.
bomewish 11mo ago
If it’s important just use b2 or hetzner storage box. Use restic or rustic for backup and dedupe and encryption. I run this setup for home and work and we’re doing this on 10tb+.
- Oxodao 11mo ago
  +1 on this, i'm running restic with resticprofile and backup to b2 it works flawlessly
aynyc 11mo ago
I was actually talking to my dad the other day. He asked me if there is a way for him to replicate his hard drive to me without touching cloud providers. The contents are family photos & videos, plus paperwork. I couldn't find a simple solution.
- carstenhag 11mo ago
  Syncthing will do a 1 to 1 connection if possible, else it will use a relay server. Traffic is encrypted. Open source. After the initial setup of marrying the devices together, it's just a matter of starting the application. Pretty much what you want?
  - aynyc 11mo ago
    Doesn't it still need a public relay server? I understand that's nitpicking, since even with tailscale, I need someone in the cloud to handle the IPs.
    - scorpioxy 10mo ago
      There are community-contributed relay servers that you can use. Check out the syncthing website, they explain how all of this works. It's a very good piece of software.
throwaway81523 11mo ago
I'm using Borg and Hetzner Storage Box. 5TB is nothing much. Big used to mean petabytes, but maybe petabytes aren't big any more either.
JustExAWS 11mo ago
Rclone with aws s3 Glacier Deep Archive storage class is about $1 a month per TB. I didn’t use rclone. I just used native AWS cli commands. But I’m an AWS guy and already had my own seldom used AWS account. Restore takes from 12 (more expensive) to 48 hours (cheaper)
hiAndrewQuinn 10mo ago
I would start by going to https://diskprices.com/?locale=us https://diskprices.com/?locale=us to anchor my price expectations accordingly.
ppg_hero 10mo ago
[dead]

slipheen 11mo ago

Like so many things, it depends- How quickly do you need to be able to restore? Is it commercial or homelab? The most cost-effective option by far would be to put a NAS device someplace offsite. You could use tailscale to connect to it remotely. After that, depending on your access patterns, either a glacier-style s3 service (aws or backblaze/etc), or a rented bare-metal server with big disks some place inexpensive.

jacobwilliamroy 11mo ago
This is for a "warm" backup as opposed to a hot backup so restoration timeline would be like seven days.

monerozcash 11mo ago

This needs more information. 5TB is far from "BIG" when it comes to cloud backups. You can probably get away with google drive+rclone+borg/restic/whatever, but it will be rather clunky. Backblaze might be a nicer backend to use. I use rsync.net with borg, but not sure about your budget. Their 1TB lifetime plan is very competitive though.

KomoD 11mo ago
> Their 1TB lifetime plan is very competitive though. I know rsync.net has been around for a long time but I'm always very skeptical of lifetime plans, you'd have to use it ~10 years for it to be worth it. (Comparing with Hetzner's 1TB storage box)

Sohcahtoa82 11mo ago

5 TB? I don't know how to count that low.

jacobwilliamroy 11mo ago
It's just bigger than any amount I've ever dealt with in my whole life (I'm 30) and also it's hard to find solutions to manage it without running into bandwidth caps and also things are running for a very long time on the client side: days or weeks.
carstenhag 11mo ago
Probably just above normal b2c backup services, and below company requirements

brudgers 11mo ago

Why are you choosing to use the cloud instead of spinning rust under your control? Or to put it another way, why is state of the art important?

kjkjadksj 11mo ago
Because you want an offsite backup of your local backups.
jacobwilliamroy 11mo ago
I need to know what people are doing recently because most of the documentation I'm finding online is from 3-5 years ago and I want the most up-to-date information.
- monerozcash 11mo ago
  People are doing pretty much exactly what they were doing 3-5 years ago, software has been pretty good for a while no so there hasn't been much change.

bomewish 11mo ago

If it’s important just use b2 or hetzner storage box. Use restic or rustic for backup and dedupe and encryption. I run this setup for home and work and we’re doing this on 10tb+.

Oxodao 11mo ago
+1 on this, i'm running restic with resticprofile and backup to b2 it works flawlessly

aynyc 11mo ago

I was actually talking to my dad the other day. He asked me if there is a way for him to replicate his hard drive to me without touching cloud providers. The contents are family photos & videos, plus paperwork. I couldn't find a simple solution.

carstenhag 11mo ago
Syncthing will do a 1 to 1 connection if possible, else it will use a relay server. Traffic is encrypted. Open source. After the initial setup of marrying the devices together, it's just a matter of starting the application. Pretty much what you want?
- aynyc 11mo ago
  Doesn't it still need a public relay server? I understand that's nitpicking, since even with tailscale, I need someone in the cloud to handle the IPs.
  - scorpioxy 10mo ago
    There are community-contributed relay servers that you can use. Check out the syncthing website, they explain how all of this works. It's a very good piece of software.

throwaway81523 11mo ago

I'm using Borg and Hetzner Storage Box. 5TB is nothing much. Big used to mean petabytes, but maybe petabytes aren't big any more either.

JustExAWS 11mo ago

Rclone with aws s3 Glacier Deep Archive storage class is about $1 a month per TB. I didn’t use rclone. I just used native AWS cli commands. But I’m an AWS guy and already had my own seldom used AWS account. Restore takes from 12 (more expensive) to 48 hours (cheaper)

hiAndrewQuinn 10mo ago

I would start by going to https://diskprices.com/?locale=us https://diskprices.com/?locale=us to anchor my price expectations accordingly.

ppg_hero 10mo ago

[dead]