4 ms·

I know all about the importance of Backups. Mirroring is not a backup. Raid is not a backup. Hard disks are not supposed to sit on a shelf unplugged for exte

I know all about the importance of Backups. Mirroring is not a backup. Raid is not a backup. Hard disks are not supposed to sit on a shelf unplugged for extended periods. Fine. Now, how the hell do I backup my 5TB of photos? :-( :-( Edit: Lots of fantastic information here https://news.ycombinator.com/item?id=7371725 https://news.ycombinator.com/item?id=7371725 Is there any service that can burn my terabytes of data onto multiple copies on "made in japan taiyo yuden" CDR? :-)

InclinedPlane 13y ago
Copy them to 2x 3TB drives, put the drives in a safe deposit box or store them in a fire safe at a friend or relative's house.
- nakedrobot2 13y ago
  Let me repeat: "Hard disks are not supposed to sit on a shelf unplugged for extended periods." This even more the case with the very large capacity disks you are mentioning here.
  - trebor 13y ago
    If you archive disks like this, you need to run software like SpinRite on them ever so often to maintain them. I'd suggest that you do it every time you update your archive on the disks.
  - jrs235 13y ago
    As mentioned elsewhere in the thread, what about burning them to 50 triple or quadruple layer Blu-ray Discs (BD-R's) then store them in a fire proof safe? http://en.wikipedia.org/wiki/Blu-ray_Disc_recordable http://en.wikipedia.org/wiki/Blu-ray_Disc_recordable
  - pjc50 13y ago
    [citation needed] A lot depends on whether you're doing a backup or an archive; if it's a backup you'll be rotating the disks in periodically.
  - InclinedPlane 13y ago
    Every month go fetch the old hard drives, copy all the new data onto them, then take the currently in use hard drives and put those back into storage. If you want more redundancy replace 2x 2x3TB drives with 2x NASes each with 3 drives in raid 5 or 4 drives in raid 1.
computer 13y ago
Print them twice and store them in different locations. Or more realistically: batch-resize the photo's so that they are much smaller, and save that version of manageable size somewhere online as a last resort for when a disaster destroys all your current full-sized copies.
- brunoqc 13y ago
  Wouldn't it be better to find ways that are not lossy?
aidos 13y ago
Start by putting a copy on Glacier. If you're on osx you can use Arq [1] to make it insanely easy. [1] www.haystacksoftware.com/arq/index.php
m_mueller 13y ago
I've come to the conclusion that Amazon Glacier is probably the first good answer since tape drives. 5TB is I think 50$ a month - if you use this data professionally, this is probably worth it. Otherwise maybe still tapes? What's the max. capacity on those these days?
joch 13y ago
I have been using CrashPlan for backing up my huge photo library, and it has been working great so far. The main benefit compared to the others is the "unlimited" cloud backup space if you sign up for their cloud service.
- nakedrobot2 13y ago
  I've tried Backblaze, but the upload speed is only a single connection and VERY slow from my part of the world. If the uploading was done via multiple connections, it would be 10X faster. As it is, I still have about 230 days left before my existing data (and that's not even all of it) will be uploaded. I will check out crashplan.
  - joch 13y ago
    The speed varies with Crashplan as well, but I usually get around a couple of megabits per second at least. I run it on my home NAS though so it's basically just set-and-forget, meaning that I don't have to remember to keep my Mac online. I just add my photos to the NAS share and it takes care of it from there.
rlpb 13y ago
> Fine. Now, how the hell do I backup my 5TB of photos? :-( :-( I use git-annex. It understands the concept of wanting multiple copies of things, and keeps track of what is where (eg. S3, Glacier, some remote rsync server, or which of my many external drives). When I want something, it gets it for me (eg. by telling me which external drive to plug in). Then, all I need to keep backed up is my git repository itself, which is tiny. I use Tarsnap for this, which means that I can keep previous snapshots without issue. > Hard disks are not supposed to sit on a shelf unplugged for extended periods. This works fine for me, when combined with some other method. Redundancy is key. And "git-annex fsck" checks a drive's integrity for me.
- nakedrobot2 13y ago
  thanks for the new possibility! Very interesting. from their page: https://git-annex.branchable.com/not/ https://git-annex.branchable.com/not/ "git-annex is not a backup system. It may be a useful component of an archival system, or a way to deliver files to a backup system. For a backup system that uses git and that git-annex supports storing data in, see bup." since you said you're using it effectively as a backup, could you please clarify what they mean, and what you mean? thanks!
  - rlpb 13y ago
    For context, I was originally answering "Fine. Now, how the hell do I backup my 5TB of photos?". My answer is that you don't need to, since photos don't usually change after they are taken. It is sufficient to simply archive them safely (and redundantly). I'm not using git-annex as a backup system. I use Tarsnap and my own tool ddar for backups. I am using git-annex to archive specific large file collections that don't ever change (eg. photos and videos). By storing these collections appropriately in multiple redundant locations, and by also backing up my git-annex repository (using the backup tools above), I have effectively "backed up" my photos. They're as safe as any backup system can make them.
jrs235 13y ago
What about burning them to 50 triple or quadruple layer Blu-ray Discs (BD-R's) then store them in a fire proof safe? http://en.wikipedia.org/wiki/Blu-ray_Disc_recordable http://en.wikipedia.org/wiki/Blu-ray_Disc_recordable UPDATE: And then I looked at pricing: As of February 2014 (approximate pricing) 4x BD-R XL disc (100 GB) US$44 each $44 times 50 = $2200. Ouch. Which would be equal to 44 months to store on Glacier at ~$50 per month (Assuming no further Glacier price reductions). I'd go with Glacier.
FreezerburnV 13y ago
It's a more expensive investment to get started, (as opposed to something like CrashPlan, Carbonite, etc.) but I definitely recommend having a Drobo product hooked up to your network, with regularly scheduled backups to it. If you got one of the five drive boxes, you could dump 5 2TB drives into it, and likely have enough capacity to store all 5TB of photos, and be able to flip on the option to have two drives crash simultaneously without you losing any data. (you definitely lose space using them, as I have a 1TB and 2TB in mine right now and only have ~900GB available) If you're using a mac, it can even act as a Time Capsule so you can direct Time Machine to back up directly to it. (my wife and I do this, it tends to back up about once a day rather than every hour like Apple promises) Yes, I know it's expensive, but it's nice to know I have a local box (nothing in the cloud!) with all my (and my wife's) data backed up to it, where if a hard drive decides to go kaput, neither of us lose anything.
- computer 13y ago
  Getting a "Drobo product" (NAS) won't do much when your house burns down or gets robbed. To be safe, you need your data to be in multiple physical locations.
  - FreezerburnV 13y ago
    That's fair enough. However, I do believe that the common case for data loss is that someone's hard drive dies, or you drop a laptop, or your laptop gets stolen, etc. Multiple physical locations provides the best safety, but there is value in having a local backup for when the hard drive dies in your computer. Also I'm not sure what (NAS) means, and I apologize if I sound like a salesman or something. Wasn't sure how to best describe a line of products made by a company without sounding like one. I was just trying to recommend a solution that I, personally, own and use to cover a pretty common case. And also feeds my slight paranoia of having a lot of my personal data and whatnot on someone else's servers.

InclinedPlane 13y ago

Copy them to 2x 3TB drives, put the drives in a safe deposit box or store them in a fire safe at a friend or relative's house.

nakedrobot2 13y ago
Let me repeat: "Hard disks are not supposed to sit on a shelf unplugged for extended periods." This even more the case with the very large capacity disks you are mentioning here.
- trebor 13y ago
  If you archive disks like this, you need to run software like SpinRite on them ever so often to maintain them. I'd suggest that you do it every time you update your archive on the disks.
- jrs235 13y ago
  As mentioned elsewhere in the thread, what about burning them to 50 triple or quadruple layer Blu-ray Discs (BD-R's) then store them in a fire proof safe? http://en.wikipedia.org/wiki/Blu-ray_Disc_recordable http://en.wikipedia.org/wiki/Blu-ray_Disc_recordable
- pjc50 13y ago
  [citation needed] A lot depends on whether you're doing a backup or an archive; if it's a backup you'll be rotating the disks in periodically.
- InclinedPlane 13y ago
  Every month go fetch the old hard drives, copy all the new data onto them, then take the currently in use hard drives and put those back into storage. If you want more redundancy replace 2x 2x3TB drives with 2x NASes each with 3 drives in raid 5 or 4 drives in raid 1.

computer 13y ago

Print them twice and store them in different locations. Or more realistically: batch-resize the photo's so that they are much smaller, and save that version of manageable size somewhere online as a last resort for when a disaster destroys all your current full-sized copies.

brunoqc 13y ago
Wouldn't it be better to find ways that are not lossy?

aidos 13y ago

Start by putting a copy on Glacier. If you're on osx you can use Arq [1] to make it insanely easy. [1] www.haystacksoftware.com/arq/index.php

m_mueller 13y ago

I've come to the conclusion that Amazon Glacier is probably the first good answer since tape drives. 5TB is I think 50$ a month - if you use this data professionally, this is probably worth it. Otherwise maybe still tapes? What's the max. capacity on those these days?

joch 13y ago

I have been using CrashPlan for backing up my huge photo library, and it has been working great so far. The main benefit compared to the others is the "unlimited" cloud backup space if you sign up for their cloud service.

nakedrobot2 13y ago
I've tried Backblaze, but the upload speed is only a single connection and VERY slow from my part of the world. If the uploading was done via multiple connections, it would be 10X faster. As it is, I still have about 230 days left before my existing data (and that's not even all of it) will be uploaded. I will check out crashplan.
- joch 13y ago
  The speed varies with Crashplan as well, but I usually get around a couple of megabits per second at least. I run it on my home NAS though so it's basically just set-and-forget, meaning that I don't have to remember to keep my Mac online. I just add my photos to the NAS share and it takes care of it from there.

rlpb 13y ago

> Fine. Now, how the hell do I backup my 5TB of photos? :-( :-( I use git-annex. It understands the concept of wanting multiple copies of things, and keeps track of what is where (eg. S3, Glacier, some remote rsync server, or which of my many external drives). When I want something, it gets it for me (eg. by telling me which external drive to plug in). Then, all I need to keep backed up is my git repository itself, which is tiny. I use Tarsnap for this, which means that I can keep previous snapshots without issue. > Hard disks are not supposed to sit on a shelf unplugged for extended periods. This works fine for me, when combined with some other method. Redundancy is key. And "git-annex fsck" checks a drive's integrity for me.

nakedrobot2 13y ago
thanks for the new possibility! Very interesting. from their page: https://git-annex.branchable.com/not/ https://git-annex.branchable.com/not/ "git-annex is not a backup system. It may be a useful component of an archival system, or a way to deliver files to a backup system. For a backup system that uses git and that git-annex supports storing data in, see bup." since you said you're using it effectively as a backup, could you please clarify what they mean, and what you mean? thanks!
- rlpb 13y ago
  For context, I was originally answering "Fine. Now, how the hell do I backup my 5TB of photos?". My answer is that you don't need to, since photos don't usually change after they are taken. It is sufficient to simply archive them safely (and redundantly). I'm not using git-annex as a backup system. I use Tarsnap and my own tool ddar for backups. I am using git-annex to archive specific large file collections that don't ever change (eg. photos and videos). By storing these collections appropriately in multiple redundant locations, and by also backing up my git-annex repository (using the backup tools above), I have effectively "backed up" my photos. They're as safe as any backup system can make them.

jrs235 13y ago

What about burning them to 50 triple or quadruple layer Blu-ray Discs (BD-R's) then store them in a fire proof safe? http://en.wikipedia.org/wiki/Blu-ray_Disc_recordable http://en.wikipedia.org/wiki/Blu-ray_Disc_recordable UPDATE: And then I looked at pricing: As of February 2014 (approximate pricing) 4x BD-R XL disc (100 GB) US$44 each $44 times 50 = $2200. Ouch. Which would be equal to 44 months to store on Glacier at ~$50 per month (Assuming no further Glacier price reductions). I'd go with Glacier.

FreezerburnV 13y ago

It's a more expensive investment to get started, (as opposed to something like CrashPlan, Carbonite, etc.) but I definitely recommend having a Drobo product hooked up to your network, with regularly scheduled backups to it. If you got one of the five drive boxes, you could dump 5 2TB drives into it, and likely have enough capacity to store all 5TB of photos, and be able to flip on the option to have two drives crash simultaneously without you losing any data. (you definitely lose space using them, as I have a 1TB and 2TB in mine right now and only have ~900GB available) If you're using a mac, it can even act as a Time Capsule so you can direct Time Machine to back up directly to it. (my wife and I do this, it tends to back up about once a day rather than every hour like Apple promises) Yes, I know it's expensive, but it's nice to know I have a local box (nothing in the cloud!) with all my (and my wife's) data backed up to it, where if a hard drive decides to go kaput, neither of us lose anything.

computer 13y ago
Getting a "Drobo product" (NAS) won't do much when your house burns down or gets robbed. To be safe, you need your data to be in multiple physical locations.
- FreezerburnV 13y ago
  That's fair enough. However, I do believe that the common case for data loss is that someone's hard drive dies, or you drop a laptop, or your laptop gets stolen, etc. Multiple physical locations provides the best safety, but there is value in having a local backup for when the hard drive dies in your computer. Also I'm not sure what (NAS) means, and I apologize if I sound like a salesman or something. Wasn't sure how to best describe a line of products made by a company without sounding like one. I was just trying to recommend a solution that I, personally, own and use to cover a pretty common case. And also feeds my slight paranoia of having a lot of my personal data and whatnot on someone else's servers.