5 ms·

I use rclone and Backblaze B2 for this. B2 used to be cheaper though not sure if that’s true with the new deep archive but it is much less fiddly and no crazy f

I use rclone and Backblaze B2 for this. B2 used to be cheaper though not sure if that’s true with the new deep archive but it is much less fiddly and no crazy fees at restore time. Rclone is also multi threaded so goes much faster compared to rsync

jwr 4y ago
Here's my "me too" — I've been happily using rclone for things like photo archives (together with my small consistency checker to check file hashes for corruption https://github.com/jwr/ccheck https://github.com/jwr/ccheck). I also use Arq Backup with B2 as the destination. This gives me very reasonable storage costs and backups I can access and test regularly.
- 369548684892826 4y ago
  "me too" - rclone with Azure Archive tier for last-resort backups. I like that rclone doesn't tie me to one storage provider, and I'm not sure why anyone would use the linked project over rclone.
  - mrich 4y ago
    As mentioned in another reply below, I found these issues with rclone, perhaps you can tell if I'm missing something: - It will not create archives by default, so potentially upload a lot of files (just like rsync). There are a few workarounds in the documentation to mitigate the cost. - It cannot do restore of Deep Archive data Edit: Can do restore, but it's a manual step.
    - 369548684892826 4y ago
      Sorry, I don't know about that, it just does what I want out of the box.
    - lathiat 4y ago
      With backblaze B2 both of those are not an issue. So not sure.
willis936 4y ago
I've been doing that via truenas scale for the past year. The upload job hangs then fails. There must be a bad permission or filename somewhere, but I don't know what the right tool is for debugging this so I've just put it off. If anything's missing it can't be large (judging from the space taken on B2).
kbumsik 4y ago
S3 Deep Archive is two time cheaper than B2. BUT you are not supposed to just rclone to S3 Deep Archive because it charges you based on the number of files you upload. The upload fee (PUT request fee) is $0.065/1000files so it will charge you a lot when you have millions of files.
- oneplane 4y ago
  Yep. Best to tar backup chunks as a whole before sending to DA.
WirelessGigabit 4y ago
B2 is $5 per TB per month, versus S3 Deep Archive ~$1 per TB per month. So if hot access isn't a concern then S3 can be 1/5th of the cost.

jwr 4y ago

Here's my "me too" — I've been happily using rclone for things like photo archives (together with my small consistency checker to check file hashes for corruption https://github.com/jwr/ccheck https://github.com/jwr/ccheck). I also use Arq Backup with B2 as the destination. This gives me very reasonable storage costs and backups I can access and test regularly.

369548684892826 4y ago
"me too" - rclone with Azure Archive tier for last-resort backups. I like that rclone doesn't tie me to one storage provider, and I'm not sure why anyone would use the linked project over rclone.
- mrich 4y ago
  As mentioned in another reply below, I found these issues with rclone, perhaps you can tell if I'm missing something: - It will not create archives by default, so potentially upload a lot of files (just like rsync). There are a few workarounds in the documentation to mitigate the cost. - It cannot do restore of Deep Archive data Edit: Can do restore, but it's a manual step.
  - 369548684892826 4y ago
    Sorry, I don't know about that, it just does what I want out of the box.
  - lathiat 4y ago
    With backblaze B2 both of those are not an issue. So not sure.

willis936 4y ago

I've been doing that via truenas scale for the past year. The upload job hangs then fails. There must be a bad permission or filename somewhere, but I don't know what the right tool is for debugging this so I've just put it off. If anything's missing it can't be large (judging from the space taken on B2).

kbumsik 4y ago

S3 Deep Archive is two time cheaper than B2. BUT you are not supposed to just rclone to S3 Deep Archive because it charges you based on the number of files you upload. The upload fee (PUT request fee) is $0.065/1000files so it will charge you a lot when you have millions of files.

oneplane 4y ago
Yep. Best to tar backup chunks as a whole before sending to DA.

WirelessGigabit 4y ago

B2 is $5 per TB per month, versus S3 Deep Archive ~$1 per TB per month. So if hot access isn't a concern then S3 can be 1/5th of the cost.