4 ms·
"If I want to recover a corrupted file, I find another copy." So you've archived two or more copies of each file? That means you're use at least twice as much
by pmoriarty 9y ago
"If I want to recover a corrupted file, I find another copy."
So you've archived two or more copies of each file? That means you're use at least twice as much space (and if you're keeping the original as well, more than twice).
For the likely corruption of the occasional single bit flip here and there, you could do a lot better by using something like par2 and/or dvdisaster (depending on what media you're archiving to).
- outworlder 9y agoYes. If your data is not in three different places it might as well not exist.
- jlgaddis 9y ago> So you've archived two or more copies of each file You haven't? It took me just one minor "data loss incident" ~20 years ago to very quickly convince me to become a lifetime member of the "backup all the things to a few different locations" club. > That means you're use at least twice as much space (and if you're keeping the original as well, more than twice). "Storage is cheap."
- pmoriarty 9y ago"You haven't?" No I don't, because it's a waste of space and money. Using par2 and/or dvdisaster I can archive a lot more files on to the same archival media and still get enough redundancy to feel secure. "Storage is cheap." Cheap is relative. Are you really buying an extra 2 TB's of storage to archive 1 TB of data, because that's what you'd need to do to archive 2 copies of each file. That's a huge waste of space and money that adds up when you're archiving a lot of data. If your needs are small or your pockets deep, you can afford to do what you're proposing, but for the rest of us who aren't made out of money it's just not practical. On the other hand, when I have worked at places which could afford to have multiple archives at various locations, I've made sure each of those archives were protected with par2 or dvdisaster, so I could recover from both rather than have one of the archives fail because of a bit flip error.
- OldSchoolJohnny 9y agoIf you have any responsibility for data protection I urge you to read literally anything on disaster recovery procedures.
- freedomben 9y agoWhat if he reads his own comment? That would be covered by the admonition to read literally anything. But my troll aside, I agree. If losing the data would cause you harm or make you sad (losing photos of your kids for example), you definitely need to have multiple backups in multiple locations, ideally controlled by different parties (so one bug on your cloud provider's side doesn't wipe out both of the copies they store for you). I've been burned by this with personal data a few times. The stakes get even higher when you are responsible for someone else's data. If they don't want to pay for the extra storage, make sure they understand the risk involved.
- jlgaddis 9y ago> Are you really buying an extra 2 TB's of storage to archive 1 TB of data ... $ sudo zpool get size zdata NAME PROPERTY VALUE SOURCE zdata size 21.8T - Yep. It's fine that you "feel secure" with your current backup regimen -- and I certainly hope you never lose any important data. After losing data once, though, I promised myself I'd do my best to make sure that it never happened again. The "primary copy" of all my data lives on the individual machines (my workstation, primarily, but there's a bit on my main laptop too) but there's also a copy of it all on a server out in the garage as well as yet another server (see above) that I have in an ISP's facility nearby. There's yet another copy of a small fraction of my files (the "really, really, really important stuff") that's sitting in AWS (via tarsnap) as well. Some folks are satisfied with a copy of their family photos copied onto a flash drive and tossed into a drawer or an external USB drive permanently sitting on the desk next to their computer. I know of several small companies in my area that thought they were safe with an external USB drive connected to their server... until they got hit with ransomware. My laptop has a pair of mirrored SSDs, my workstation has a pair of mirrored SSDs and a pair of mirrored "spinners". The server in the garage (my "first backup") has RAID10. That box at the ISP has mirrored SSDs plus a "raidz2" that the backups live on. Some of us just want a little bit more reassurance than others. :-)
- rikkus 9y agoStorage is cheap indeed, though it takes some effort to make it cheap. 99% of the digital data I'm keeping for the long term is family photos and videos. All my photos go to Dropbox (easy copy-from-device and access anywhere) and are then backed up to multiple locations by CrashPlan. It'll be a while yet, but in the next few years I'll be hitting the 1TB Dropbox limit. I'm hoping that Dropbox make a >1TB 'consumer' plan in the next couple of years. There's no way I'm assuming my backups are fine, deleting from Dropbox to make space, then finding out in a few years that some set of photos is missing. I also sync up to Google Drive - but again, there's a 1TB limit (or a large cost). In the future, I might have to create a new Dropbox account and keep the old one running. Storage might be cheap, but keeping it cheap is tricky.
- joombaga 9y agoAre you using the small business version of CrashPlan? I was using CP too until they discontinued their B2C.
- SyneRyder 9y agoSame here - I'm currently still migrating from CrashPlan B2C to using Arq backing up to Backblaze B2. (Being able to access B2 from Panic's Transmit Mac app made B2 really attractive to me as well, and it looks like I'll save a lot of money compared with CrashPlan.)
- joombaga 9y agoI'll look into this. Thanks for the info!
- viraptor 9y ago> Storage might be cheap, but keeping it cheap is tricky. If it's really for pure backup, not continuous sync, Glacier is $4 per TB.
- imtringued 9y ago
- tedunangst 9y agoIf you're using par2, I'd say that's closer to recovering a second copy than trying to extract meaningful data from a corrupted file. (The internal structure of the format is irrelevant, thus concerns about it are outdated.)
- pmoriarty 9y agoEven when using par2, the fact that xz files contain no version number for the format is still troubling.
- f4rker 9y ago>So you've archived two or more copies of each file? If youre not keeping more than 1 copy of important files what are you even doing?
- technion 9y agoI would generally suggest you're more likely to corrupt/lose your whole backup than to have one corrupted bitflip not addressed by the filesystem or underlying storage.