8 ms·
Ask HN: How do you plan to archive all your digital content?
I have been wondering on how I could preserve some of my digital photos so my son could access them when I am long gone. I currently have a mix of iCloud and OneDrive but those requires account logins and I couldn't find a convenient way of like hey son here is all your baby pictures and all of our adventure when we where still living together. Any suggestions?
- epc 4y agoI have two pocket SSDs that I archive to on an irregular basis (basically quarterly) of selected documents and photos. Each is in a small faraday bag, whichever one is in the house is in a safe, the other is in a safe deposit box at my bank. The drives themselves are unencrypted but some of the documents are encrypted (e.g. I export my 1Password vaults quarterly and encrypt those). I don’t have any kids nor plans to have any but have lost about 30% of the digital stuff I thought I was keeping safe (through bit rot, fading CDRs, regrettable document retention policies I was unaware of). My advice would be to cut selected items to an SSD on a routine basis and either stash it in the family safe or wherever you keep important files. Use an existing filesystem, not whatever the latest newfangled vendor specific one, and use a fairly well documented format for images. Store documents as text or rtf.
- danparsonson 4y agoJust be aware that SSDs are not stable long-term stores - they need to be periodically powered up or they will eventually lose data.
- epc 4y agoAm hoping that monthly/quarterly use is enough but that’s a good point to research.
- danparsonson 4y agoFrom my own experience that's fine - I mentioned it more as a general note for anyone thinking of using SSDs for archival. I know someone who left a computer unplugged for a couple of years who lost all their data; I myself have successfully stored SSDs for a year or so without issue. Anyway as you say, worth doing your own research.
- mxuribe 4y ago> ...Each is in a small faraday bag... Now that is a great idea! I do something similar to you with the exception of using faraday bags. Thanks for sharing!
- hamsterbase 4y agoE-Books:I use zotero to store all my books and notes. Using webdav to back up data to nas. Photos: I will manually copy the photos from the drone and DSLR to the nas and format the TF card after copying. Phone photos: I use the software provided by my nas service provider to back up my photos Web pages: I run my own docker version hamsterbase on nas. Store all the pages I am interested in.
- mikece 4y agoNAS and SSD, multiple copies. I don't know if I'll get to the point of archiving on Amazon Glacier or something similar but that's always an option. Having just gone through a funeral for my dad this summer I can say that having non-digital images (photo prints, negatives, slides) doesn't always yield the most durable image. Compared to the possibility of storage devices failing I think digital images will be far more durable than previous analog formats... having those in a format and location where the next generation will know to find them is another matter. (For example: how long before the images I burned to archival quality DVDs cannot be viewed because no computers ship with optical drives and external optical drives are getting harder and harder to find?)
- hatware 4y ago3-2-1 backups haven't let the world down.
- superkuh 4y agoKeep it simple. No raid or weird filesystems, no special NAS computers, no encryption, no remote services. I stuff every computer I assemble with hard drives until it can support no more then mirror important data from every computer to every other computer with rsync.
- whalesalad 4y agocaveman-edition ceph
- irrational 4y agoBut, are all those computers in the same location? Just one fire away from losing everything? I have all my photos and videos on local storage, but it is all mirrored to a secure cloud storage service that costs hardly anything. I’ve pulled the backups from that cloud service (you can’t access the files directly) to make sure it works properly.
- throwamon 4y ago> a secure cloud storage service that costs hardly anything Which is...?
- irrational 4y agoCrashplan I’ve also heard good things about Backblaze.
- mxuribe 4y agoI'm really curious to learn which cloud service that you use which "costs hardly anything". Would you kindly share?
- irrational 4y agoCrashplan. I’ve also heard good things about Backblaze.
- WesleyLivesay 4y agoI think the best way to hand that stuff off to your children is to wait until they get to a point where they hand off can happen, when they are adults, and then hand them a highly curated set of photos on a USB drive and let them do with it what they will. The key is in the curation though, it is unlikely that your child will want to sort through thousands of blurry, duplicate, or unrelated photos. 100 really good and special photos/videos are better than a mixed up mass of 5,000. If you are concerned with passing away before your son is of age, just make sure there are multiple backups, and the sign in information for anything that isn't local is communicated to his guardian.
- gwbas1c 4y ago> The key is in the curation though, it is unlikely that your child will want to sort through thousands of blurry, duplicate, or unrelated photos. 100 really good and special photos/videos are better than a mixed up mass of 5,000. Honestly, I would really enjoy sifting through 5,000 photos from my grandparents or parents. (Sometimes the best photos are the embarrassing or private photos, like the time my dad asked me to help back up photos and I saw my mother trying to recreate a famous album cover.) I appreciate the curation too.
- ngetchell 4y agoI think the act of developing film meant that photographs had a higher signal to noise/duplicate/blurry factor. It's trivial to take 100 pictures in an afternoon, with only 1-2 being worth saving.
- ryanmercer 4y agoBy letting it go. I used to horde, multiple formats and offsite... then I just let it all go. I've got a local copy of files on my machines plus a backblaze copy, that's it, I don't need tens of terabytes of data I'm almost certainly never going to use.
- humblepie 4y agoI use PhotoSync on my iPhone to copy the original photos from iCloud to my Mac. Then I have two 5TB LaCie Rugged drives that I just attach to the Mac. Finally, using ChronoSync to schedule copying contents from the Mac to both drives as mirrors.
- seszett 4y agoI simply print the photos I want to keep, and put them in photo albums like my parents and grandparents did before me. I don't expect my children to be able to explore my digital archives when I'm dead in 50 years or so, so anything I want to be found then I print or otherwise place on a physical medium that's readable without computers. The less important stuff can stay archived on hard disks as plain files on a FAT32 filesystem. With a physical label on the disk describing the contents.
- SoftTalker 4y agoYeah and I think people overestimate how much interest their kids will have in old photos of their ancestors. I cleaned out boxes of photos from my late parents house. Lots of pics from my childhood years, somewhat sparser selection of older photos. Never look at any of them. I have a framed photo of my parents on a shelf; that's enough. I also basically stopped taking photos of stuff about 15 years ago when I realized I never go back and look at them again. Bottom line, I don't have much digital content to archive, and I plan to keep it that way.
- yamtaddle 4y agoPro tip to parents out there: Your kids will want to see other people in those childhood photos, more than they'll want to see 100,000 photos of their own faces. You, when you were younger; their now-gone grandparents; that friend they had for two grades before one of them moved ("OMG I totally forgot about them!"), that kind of thing. You'll get way more than enough photos of your kids, so far as your kids themselves are concerned, without even trying—it's the others you may neglect. Now, for your own purposes, yeah, you probably want the photos of your kids more than anyone else who might get in the shots—but try to think of them and snap one or two of other people in the room, when you're taking e.g. birthday photos, or pan around from time to time when shooting videos of them. They'll eventually get more enjoyment out of the stuff, if you do.
- KAMSPioneer 4y agoCounterpoint, your grandchildren will appreciate pictures of their parents as kids.
- chris-buck 4y agoI've installed iCloud on my Windows box, with iCloud photos enabled. Then I right-click the iCloud Photos folder and select "Always keep on device." It takes a bit but everything gets downloaded. I then have a script that makes a zip of all photos monthly with a timestamp for the title.
- Apreche 4y agoI have all my private data backed up appropriately on a NAS, in the cloud, etc. When I'm gone, whoever has access to my accounts, inherits my stuff, buys my property an estate sale, etc. can do with it whatever they please. However, for the digital content I have published, such as our podcast, I plan to form and fund some sort of trust that will be tasked with keeping it online and publicly available for as long as possible. This may just mean making sure it is archived in at least one library, the Internet Archive, etc.
- memorable 4y agoI am pretty simple when it comes to archiving data. I archive my blog posts and writings using archive.ph. For storing other type of content (images, etc.), I use Icedrive[0], which provides enough storage for me to store important images and other data. [0]: https://icedrive.net https://icedrive.net
- david_p 4y agoresilio sync (aka BTSync) on my iPhone to send all pictures to a dedicated server in the cloud, where I store a copy of all my stuff. Format: a simple folder tree /Pictures/YYYY/YYYY-MM-DD-Album-name
- jefc1111 4y agoI recently zipped all my photos ever into one ~400gb file (along with a .txt file list). I have the photos 'online' at home and a copy of the zip archive at home, but the cloud copy is in AWS where I pay about $0.40 a month to keep it there in Glacier Deep Archive. i plan to repeat the exercise every 2 or 3 years, building up a set of zip files. At that price i can treat it like a 'fire and forget' dustbin of stuff that I'll probably never have to access...
- atemerev 4y agoPrinting photos is the best way, I think (if stored in a dark place). There are also M-DISC disks and drives (archival quality BluRay with some specialized layers chemistry). Since they are positioned for long-term archival, I expect there will be some need to read them in the future, so some sort of drives will remain available.
- rektide 4y agoIm surprised more clouds dont let folks put credit into their accounts. I used a smallish openstsck based provider LunaNode for a little while that had a "wallet" one had to pre-pay into... conceivably one could just pay-in-advance the estimated rate for the next couple hundred year. You'd have to find a provider you trust to stay around. We can iterate from there. Wouldnt it be sweet to be able to let other people dump more credit into your provider wallet, so it could keep going? Maybe we could standardize a "end of service" webhook/notification, and your website & content could jump ship & re-instamce itself on another provider automatically if your current one declares it's going under. Dont forget to setup after-life DNS too!
- scarface74 4y agoAnd what happens when LunaNode goes out of business or gets acquired and the CEO posts an “Our Amazing Journey” letter on their website?
- JohnFen 4y agoI just keep them on a local fileserver, and do backups of that as normal.
- irrational 4y agoLocal, as in your own home? What happens when the home burns down and you aren’t there to grad the backups?
- soco 4y agoSame local NAS, where backup = encrypted in the cloud, and also on M-Disc blu-rays kept in my office (where I also keep two BD units which I test like once a year). But I love the other idea of printing the best out, I'll get to that for sure.
- vorpalhex 4y agoHomelab with ZFS and ~60tb racked. It backs up to a second box in the house. I occasionally swap drives with a nearby friend. Homelab runs photoprism (open source google photos alternative), plex, code-server, etc.
- scarface74 4y agoAnd what happens if your house catches fire?
- blitzar 4y ago> I occasionally swap drives with a nearby friend
- NoboruWataya 4y agoOn this topic, does anyone have recommendations for good and relatively affordable home NAS hardware? A lot of the ones you can buy with all the bells and whistles included are quite expensive and the cheaper ones out there look kind of shoddy. I bought a little kit for turning a Raspberry Pi into a NAS device using a SATA hat but the connection seems dodgy or something as it doesn't recognise the hard drives persistently. Shame because it looked quite good.
- bick_nyers 4y agoYou can check out the data hoarder subreddit, which often has good suggestions. My current setup is Ryzen on an AsRock motherboard (let's me use Unbuffered ECC RAM), and you use LSI HBA cards (see art of server on EBAY) which gives you more SATA ports. Still in the range of $500-$1000 (no HDD) depending on how you spec it out, but far more powerful and expandable for anything even close to that price range. If you are going for something in the $200-300 range, you can likely take the same path, but will make compromises in e.g. motherboard choice that won't scale well over time (but can still do a basic 32TB file server with ease).
- aborsy 4y agoWould you mind posting your AsRock hardware? I was considering AsRock x300 mini desktop, but it supports only 2.5” HDDs. I am looking for two 3.5” HDDs, one NVMe and ECC RAM, low power CPU, around $600 or so.
- bick_nyers 4y agoI am using the B550 Steel Legend (6 full speed SATA ports, which is perfect for raidz2 in ZFS), a Ryzen 5800x (you can't use the g variants with Unbuffered ECC), and my RAM is Kingston ECC Unbuffered DDR4 3200 Model KSM32ES8/16ME. You will need a dedicated graphics card, thankfully they are going down in price, if you are going to run Plex/any kind of transcoding, I recommend looking up "NVENC Encoder Matrix" and finding a card that support h265 and B-Frames (Turing or later). Biggest cost is easily the CPU, a 5600x will run you $190. You could try a 3600 for $150 but I have not tested that with the ECC RAM, which is completely undocumented what kind of support is there it seems. You probably want the 5600x anyways, I was able to overclock the ECC RAM up to 3866MHz and set my infinity fabric FCLK to 1933MHz (the 3600 can't reach that really at all). It's only like 5% more performance though. Might need to look at used parts (Mobo, CPU, GPU) to get it for $600, but I haven't looked up prices and tallied it up.
- pak9rabid 4y agoI have a RAID5 over a set of 3 internal SATA drives, then an external enclosure with 5 drives (RAID5) for backup connected to the same machine via eSATA, where an rsync job kicks off every night to sync between primary/backup volumes. No off-site backup, though, which I know is a single point of failure which I hope to remedy at some point.
- sriku 4y agoI've been trying a different way of recording for several years now. Have no way of knowing whether it will be better or worse than photos, but ... Honestly don't care because it is fun for me. When some interesting thing happens with the kids, I shoot an email to their accounts in which I write the conversation, the way I felt, how we admired what they did, what we learnt from them and such things.
- wahnfrieden 4y agoArdrive is interesting in this space: distributed storage with the promise of one-time payment sufficient to cover storage cost permanently. Every other solution I’ve seen requires ongoing rent, or ongoing stakeholder interest+capability for preservation without financial incentive
- maerF0x0 4y agoI was sad to learn that many of my IG stories (and presumably applies to reels too) have broken music links (even though the item is still in the library) . That means I cannot rely on IG to adequately safeguard my memories.
- JohnFen 4y agoYou shouldn't rely on any company to adequately safeguard your important data in the long term.
- WalterBright 4y agoI have media going back 50 years. A common theme is all of it becomes unreadable after a while - not because the media degrades, but because you can't get a device that can read them (or it becomes very expensive to obtain such a device). Punch cards, paper tape, magtapes, 8" floppies, 5.25" floppies, PC hard drives, cartridge tapes, DECtapes, zip discs, hard drives, etc. The easiest by far solution is to simply copy them forward to new media once every year. Have an offsite copy. I don't trust any cloud backup. They can go dark at any time for any reason, you can lose your password, you can get "canceled", a hacker can ransomware it, etc. I periodically check that ransomware hasn't infected my backups by seeing if an unrelated device, like a media server box, can read the files on it. I have no faith in "archival quality" media, as if you can't find a device to read it, it's worthless. Have you noticed that computers increasingly don't come with CD drives anymore? Why would anyone want them, since a 64Gb USB stick goes for $6? Soon there will be no way to read those "archival quality" CDs.
- WalterBright 4y agoIn going through my Dad's papers, I found a listing of the first game I wrote (in DECSystem 10 Basic). I thought that was lost in the mists of time. It was fun reading the ancient code. I scanned it and put it on my backup drive. I threw away my old punch cards, paper tape, magtape, DECtape, etc., as it wasn't worth the effort to find a machine that could read them.
- PaulKeeble 4y ago3-2-1 seems to be a reasonable strategy for keeping the data relatively safe over the long time and maintaining some offsite backup saves it in the event of house fires and the cloud is convenient for that it just can't be the only storage. But the bigger issue is often conversion of data, there is often a window in which a format can be converted from an old format to a newer or more popular one and you often have to assess that yearly.
- WalterBright 4y agoI tend to archive things in the most common formats, and haven't had much trouble with it. For example, in the 80s I'd use various compression programs. When I eventually archived them, I uncompressed them with the old compression programs. I store pdfs with a .txt version of their contents, too.
- causi 4y agoA bigger question is indexing that content. I would pay good money for a software suite that would let me scan all my old family photos and attach both text and audio. I'd love to sit down with my mother and one-by-one go through them and let her record the story behind them. I don't understand why such a format doesn't exist.
- noSyncCloud 4y agoPut them in your PowerPoint alternative of choice and record narration?
- causi 4y agoThat doesn't let you organize them by topic or by the names of people present in the photos, though. If I wanted to find "every picture with grandma in it" I'd be SOL.
- quaffapint 4y ago1. I've converted any old media format to digital 2. I back up locally to a drive inside a firebox (w/ built-in usb connector) 3. Backup offsite encrypted to Backblaze b2
- neilv 4y agoIf you're ready to share all that, and it's just family photos&videos (nothing too sensitive that you won't mind them leaking through ordinary bad security and social media), then you could give them access as soon as they have their own online accounts, and they could just grow up always accessing it. But, for financial/estate data that my heirs might eventually needed, I have a script that `rsync`s select data to (multiple copies of) an unencrypted external USB drive, in FAT32 format. Included on the drives, and also stored printed with them, is a one-page `READ-THIS-FIRST.txt` file that says what info is on the drive, as well as top-level summary about other things they might need to know (like what insurances I have, any will, and how to access the GnuCash files). There's also a PDF&paper one-pager assets&liabilities report from GnuCash, for use before anyone gets around to installing GnuCash. This enumerates all the accounts where money is stored or owed, and sets expectations for the reader's very-very tiny inheritance. There's also passwords. Mine are currently handwritten in a redundant pair of Rite In The Rain paper notebooks, one off-site. But you could also use a password manager service, and some services can release more passwords to family members upon your expiration. Your survivors being able to just one of those accounts, might save them a lot of headache, and you don't know which one that will be. Even for only family photos that they can already access, you even might want to have a one-pager bus-factor document that includes things like how to keep the photos from disappearing (such as by not getting locked out of access, not failing to pay a cloud service bill, or not donating the old junk servers that were storing them). If this one-pager is accessible to kids, maybe structure it as notes for yourself of how things are set up, like an engineering nerd, without alluding too much that they'll someday need to do it.
- aborsy 4y agoCold storage may not be a great idea. With warm storage, you can run frequent checks like scrubs. Storing data with two cloud providers is a good option. But Online storage must be encrypted. So the question is, what backup or encryption software do you use to be able to decrypt the data in 10—15 years or so?
- tennisflyi 4y agoLocally.