3 ms·

Ask HN: Need advice for long-term data archival

7 points by samch 12y ago

A colleague of mine has terminal brain cancer and is a single mother of two very young kids. She has recorded a lot of video (4k resolution) and needs a way to preserve that data for her children to watch as they grow up. I am familiar with a number of online storage options from Amazon's Glacier to Dropbox, and I've even thought about having her use YouTube. I'm sure there is a small monthly budget that will be available to pay for this, but the solution cannot be too expensive. I would also like for the solution to be easy enough for her kids to access. I'm really hoping to find a service that will be around in the long term - maybe 20 years or so - until her kids are college age and can manage this content on their own. How should I advise her to pick a service? What options do the HN readers recommend? Is there a service out there that is better suited to type of use case than others? Thanks in advance.

brudgers 12y ago
Twenty years is a really long time. In 1995 the choices would have been things like WORM drives, DAT, VHS, Floppies, IDE hard disks, etc. Of those, VHS or Floppy are probably the most likely to be accessible today. And it would be painful. I would suggest multiple backups each in multiple formats. I'd look at .ISO formats in both several hard copy and cloud archives. And more native formats in the cloud. The key is to have a team that is willing and capable of stewarding the material as technology changes and services break down. My best wishes for all affected.
- samch 12y ago
  The multiple backup solution is the one we're going with for now. I just met with her this week, and she is using an external RAID-1 solution while she is in the process of recording the videos to leave for her children. I'm also actively working with her to select a could provider to host an additional backup. Her family will need to make a fresh copy of the data every 5 years or so as magnetic media is only so reliable. I told her that I would, of course, make myself available to help with that process. I'm just not sure if they'll remember to do it. Thanks for your well-wishes.
trcollinson 12y ago
As I have mentioned on HN before I have been working in video and image data quite a bit over the last few years and, frankly, that's a LONG time. We usually think 18 to 36 months out. 20 years, now that's a problem. I honestly don't think any of the current solutions that are out there will work in 20 years without numerous iterations of upgrades in the mean time. With that being said, I am quite willing to throw my own personal time and energy into coming up with a long term solution for this family. If you are interested, my contact information is in my profile. Feel free to reach out.
- samch 12y ago
  We work at Duke University, and I've reached our to the people who work at the University Archives. They basically mentioned the same thing you did about 20 years being a long time. As a software developer, I have thought about what it might take to create a system to support this type of use case using various cloud providers on the backend. Thanks for your offer of assistance. I will certainly let you know should that need arise.
vrypan 12y ago
Maybe http://www.longaccess.com/ http://www.longaccess.com/ is what they are looking for.

7 points by samch 12y ago

brudgers 12y ago

Twenty years is a really long time. In 1995 the choices would have been things like WORM drives, DAT, VHS, Floppies, IDE hard disks, etc. Of those, VHS or Floppy are probably the most likely to be accessible today. And it would be painful. I would suggest multiple backups each in multiple formats. I'd look at .ISO formats in both several hard copy and cloud archives. And more native formats in the cloud. The key is to have a team that is willing and capable of stewarding the material as technology changes and services break down. My best wishes for all affected.

samch 12y ago
The multiple backup solution is the one we're going with for now. I just met with her this week, and she is using an external RAID-1 solution while she is in the process of recording the videos to leave for her children. I'm also actively working with her to select a could provider to host an additional backup. Her family will need to make a fresh copy of the data every 5 years or so as magnetic media is only so reliable. I told her that I would, of course, make myself available to help with that process. I'm just not sure if they'll remember to do it. Thanks for your well-wishes.

trcollinson 12y ago

As I have mentioned on HN before I have been working in video and image data quite a bit over the last few years and, frankly, that's a LONG time. We usually think 18 to 36 months out. 20 years, now that's a problem. I honestly don't think any of the current solutions that are out there will work in 20 years without numerous iterations of upgrades in the mean time. With that being said, I am quite willing to throw my own personal time and energy into coming up with a long term solution for this family. If you are interested, my contact information is in my profile. Feel free to reach out.

samch 12y ago
We work at Duke University, and I've reached our to the people who work at the University Archives. They basically mentioned the same thing you did about 20 years being a long time. As a software developer, I have thought about what it might take to create a system to support this type of use case using various cloud providers on the backend. Thanks for your offer of assistance. I will certainly let you know should that need arise.

vrypan 12y ago

Maybe http://www.longaccess.com/ http://www.longaccess.com/ is what they are looking for.