7 ms·
Ask HN: How do you back up your drives?
I'm currently going through a meltdown because my Drobo, which is supposed to give users "redundant protection", has corrupted +4TB of data: years of photos, all my music, all docs that go from way back at university to personal projects, raw video files... everything. I just want to know what HN users use to backup their data, drives, etc. I haven't heard good things from Backblaze, Crashplan, etc, so I was wondering if there are other alternatives. Thanks
- davidbarker 12y agoI currently use Backblaze, and backup to S3 using Arq. Most of my work is stored on Dropbox, too.
- Errorcod3 12y agoI bought a external drive and backup all my computers to that. Better yet if you can store it in a water/fire proof safe. You could also look at setting up a NAS. http://en.wikipedia.org/wiki/Network-attached_storage http://en.wikipedia.org/wiki/Network-attached_storage I don't' trust any backup source that is not my own.
- pwg 12y agoLocal linux machine with three 2T drives (WD Red's), setup in RAID 5 for redundancy against a drive failing. LVM running on top of the RAID 5 drive trio to allow slice-and-dice allocation of the space. The LVM backup partition only consumes 220G. rsnapshot (http://www.rsnapshot.org/ http://www.rsnapshot.org/) from other machines (they are Linux as well) onto the Linux backup server (so now there are two physically separate copies, master machine and backup machine). And rsnapshot provides "point in time" views of the backups. Periodically rsync the rsnapshot tree to an offsite location with enough space to hold the 220G worth of backup data. What I don't do, which you imply you did with your Drobo, is have the only copy of anything truly important on the RAID5 array only. If Linux md or LVM corrupts the backup partition I just remake it, and start rsync'ing again (and pull old snapshots from the offsite). If one drive fails in the RAID5 array, I'll replace the drive and rebuild the array, but the data should still be there. If two drives fail in the RAID5 array close on each others heels (so that the whole RAID array goes away) then this is the same as if md or LVM corrupted the data. "Backup" does not mean having "redundant protection" in one device, it means having plural separate, fully independent, copies such that corruption/loss of one copy is meaningless to the other copies.
- fernandotakai 12y ago* work and dotfiles are on github * personal projects are both on github and on tarsnap * my pictures are on google drive and on tarsnap * my music goes into google music (even thought now i'm using spotify now) everything else is either on google drive or in any other kind of cloud service.
- dewey 12y agoServers: - Configs and small important files: http://www.tarsnap.com/ http://www.tarsnap.com/ Laptop: - 2 x TimeMachine - Amazon Glacier with Arq (Weekly, just important files) - External Disks mirrored to same model with CarbonCopyCloner
- DanBC 12y agoI list the stuff that I'd be really distressed to lose (the photos of my child; My Great Novel;). That stuff gets archived (zero compression) with recovery data; with PAR2 recovery data as well; it then gets copied to multiple USB sticks and microSD cards. These get posted out to various family members (who are asked nicely to keep them somewhere safe) and kept in my firesafe. Any passwords and serial numbers and etc are printed out, with the name and URL of the software they're for. A copy of that goes in the fire safe. There are copies of the spreadsheet on various drives. All the other stuff is kruft. I do have multiple copies of my music but that's only because tagging MP3 files is hell. I'd be interested to hear how the drobo failed.
- larrys 12y agoLittle tip for the fire safe. YMMV. Put some cash in the fire safe. If someone breaks into it (thinking there is money) they will probably be less likely to take other things in the safe or destroy those items (I mean a hard drive they might leave - obviously they will take anything of clear value). Assumption also is that the HD is encrypted. This comes via a practice a long time ago with cash registers. You leave the cash register unlocked and put some cash in it. Thief is presumably happy and doesn't trash the rest of the place. "Any passwords and serial numbers and etc are printed out, with the name and URL of the software they're for. A copy of that goes in the fire safe. " Doesn't seem like a good idea to put plain text passwords (is that what you are saying) in a place that someone will look thinking there is something else of value in there.
- Spooky23 12y agoMake sure the fire safe is rated for data. Most are not. The typical fire safe is meant to prevent auto-ignition of paper. That means that your media could easily be subjected to temps up to 400 F -- maybe higher! You've controlled for this by sending them offsite as well, but most ppl won't!
- mrgriscom 12y agoCrashplan. Off-site backups. Works great on linux. Saves old versions/deleted files forever. Multiple devices for cheap. Set it and forget it.
- kooshball 12y agoI really want to setup something that's easy to use for my parents. Does anyone have any suggestions? Crashplan seems like the easiest choice. Does backing to to glacier with arq require any technical expertise after it's all setup?
- dewey 12y agoNo, it's all automated and works fine without checking up on it. The only thing you have to do from time to time is updating Arq. You could also enable mail notifications so you'll get notifications if Arq reports any errors.
- hfreire 12y agoI've been using Degoo for a while. They encrypt all your stuff and spread it around the globe. You get the first 100GB for free + 50GB when you invite friends. http://www.degoo.com http://www.degoo.com
- chdir 12y agoCaution : bad ratings on WOT : https://www.mywot.com/en/scorecard/degoo.com https://www.mywot.com/en/scorecard/degoo.com
- deleted 12y ago[deleted]
- Yrlec 12y agoCo-founder of Degoo here. I've tried to address the concerns the reviewers at WOT had but unfortunately they refused to change the factually incorrect statements in the reviews. For instance, they claim that we are inactive and only collect e-mails but you can see at our website that we are very much active. You can see our discussion here: https://www.mywot.com/en/forum/20714-degoo-com https://www.mywot.com/en/forum/20714-degoo-com I'd love to answer any questions or concerns you might have about our service.
- deleted 12y ago[deleted]
- geekam 12y agoNot very happy with this but I have 2 (1T) hard drives which I mirror via rsync and back up on those. I have 1 (1T) hard drive at work, to have an off-site backup. Plan to use Arq for important files to back on Glacier. I'd change the two hard drive system to a 4 bay RAID setup using Synology etc. ---- I was thinking of buying Drobo but I guess I was always scared of their "secret sauce" and getting stuck with recovery if anything happened to it.
- ja27 12y agoThat's similar to what I do. I have a pair of 2TB bare SATA drives that I rotate offsite. I got two different brands to try to minimize the possibilities of them both failing at nearly the same time. I used to use Synctoy on Windows to back up to them but now it's either cp or rsync from my MacBook. I do need to do this more often and I really should upgrade to bigger drives now, but they hold the stuff I'm most worried about losing. What I'm most worried about is if my original source files got corrupted, I don't want to back them up over top of good copies. I've tried online backup services and was really disappointed by the throughput. I think it took me nearly 5 months to do an initial backup on Crashplan and that didn't even include my photo archive, VMWare images, etc. I've tried using services like Flickr as photo backup but the retrieval process is brutal. There's no good way to bulk retrieve photos from them. I do use Dropbox to sort of back up anything I'm currently working on but they don't retain history past 30 days without paying for Packrat. Plus I've soured on them due to privacy concerns. I do use BitBucket for almost all of my source code and I've been pretty happy with it. I do use Time Machine on an old 1TB drive for my main system (MacBook Pro) and that's actually been pretty damn good. The handful of times I've needed to recover a file lately, I've been able to get it from Time Machine in a couple minutes with little hassle. I do need to get a newer, bigger drive for that sometime.
- geekam 12y ago>> I got two different brands to try to minimize the possibilities of them both failing at nearly the same time. Wow! I forgot to mention but I do this too! Thank you! My wife has called me paranoid of the highest order and now I can point to this comment and feel better that I am not the only one. >> I've tried online backup services and was really disappointed by the throughput Yes. I have been disappointed with all of these services. >> Flickr as photo backup but the retrieval process is brutal. Yes. I tried apps like Flickery etc but it is just not worth it. >> I do use Dropbox Nice idea. I have some files on it but I only use Dropbox (the free version) for files that I need on-the-go for anywhere-access. >> I do use BitBucket for almost all of my source code and I've been pretty happy with it. Same here. Bitbucket for some private random experiments and projects, Github for things that I don't care about. >> I do use Time Machine on an old 1TB Actually, Time Machine is the only decent backup utility I have ever found. Ever. The ease of use is just mind-blowing. I remember more than a decade ago, when my only chance was to wipe out the system and reinstall the OS or to keep a partition or separate drive for home/data. And with newer OSX, Time Machine can back up to as many drives as you want, which makes it easy for me to take actual direct backups on multiple drives. On top of it, privacy is such a big concern. Then, it usually becomes a pain. I tried https://www.aerofs.com/ https://www.aerofs.com/ too but then left it because it was too buggy. I will have to try it again.
- owenwil 12y agoWhat have you heard bad about Backblaze? I backup my two machines there and have nothing but praise for them. It pushes up new files as soon as they are saved and they can post you a hard drive with your files if you need to do a restore.
- Spooky23 12y agoYou need to be careful with their excluded file lists. On the past they have changed them and its easy to miss whatever notification they send. Also, in my experience their throttling sucks. If I get back from traveling with a bunch of data like pictures or video, it crushes my upstream bandwidth and disrupts VoIP. That said, for the price, it is an excellent service, and their restore mechanism works well.
- carbocation 12y agoLocally, I use TimeCapsule on a standalone drive networked over my router. This happens automatically when I am on my home network. I also duplicate things to Google Drive. This happens automatically when I am on any network.
- larrys 12y ago"I use TimeCapsule on a standalone drive networked over my router." Doesn't protect against a fire. Or someone who swipes the router and/or the data source. Suggest either an offsite backup or a fireproof hard drive (iosafe for example) to backup the backup. (TC is funky with attached drives or at least that is what I have found when trying to use an attached drive on an airport extreme). (Of course someone can also swipe the fireproof hard drive).
- carbocation 12y agoI agree that TC is funky with attached drives. Also, I don't care (much) about old versions of my personal data. If I did, this would be an unacceptable backup solution because fire/theft of my TC could wipe it out, as you mention. But between my TimeCapsule and my Google Drive, it strikes me as unlikely that my laptop, my TC, and my Google Drive will fail me all at the same moment. If so, I would be SOL.
- k-mcgrady 12y ago>> "I haven't heard good things from Backblaze" I've got nothing but praise for Backblaze. Been using them for about 4 years and it's worked flawlessly.
- ricardobeat 12y agoExcept it doesn't really keep backups for external drives, unless they are connected to your machine at least once every 30 days.
- k-mcgrady 12y agoDidn't know that as I only use one internal drive, no externals.
- sandis 12y agoAmazon Glacier for stuff that can't be redownloaded or reproduced - personal photos and videos, mostly. Time Machine for documents and projects - those usually live in Dropbox or Github/Bitbucket as well. I don't bother with backing up OS/configuration, since it takes no more than a day to get a clean OS and applications to a state where I'm feeling at home.
- msherry 12y agoTime Machine to a local Synology Diskstation w/RAID, plus Backblaze. Will also chime in that I have nothing but love for Backblaze -- every time I've needed them (small files only, never a full restore yet), they've worked flawlessly.
- Spooky23 12y agoEasiest: Backblaze Cheapest for most people: Arq Nerdiest: rsync.net
- ArkyBeagle 12y agoIt might be worth having a set of n humble USB drives, one of which is in a safety deposit box ( cycle through them ) with a weekly copy of everything on the RAID. I set n at 2, and substitute a cheap "fire safe" for the safety deposit box ( I know, I know... ) . I don't know that I will build another RAID when this one goes. I have yet to pull anything off it. But my primary copy of things is all still on my main machine, un-RAIDed. For legacy software reasons I won't bore you with, that's still a Win7 box.
- ggreer 12y agoI think every child should experience a disk failure. After all, it's only a matter of time before each of us has one. Better to get it out of the way when we have little to lose. People replying will recommend many backup solutions. No matter what you go with, it's important to trust no single party. Your home could burn down. Dropbox/Crashplan/S3/etc could fail. Media in a safe-deposit box could become corrupted over time. (This is common with flash-based storage and CDs.) Pick at least two independent backup solutions. Also, backups are a lifestyle change. They're something you have to maintain indefinitely. File systems, I/O interfaces, media formats, APIs, and businesses all come and go over time. There's no one thing that is sure to keep your data around for decades. Finally, most people don't have much data that's actually important. I have maybe 20GB of stuff that matters to me: photos I've taken, documents I've written, and a few videos. In terms of value per byte, documents are the highest, followed by photos, then video. It's much easier to maintain backups when your data set is small. Music, books, and other mass media can be re-bought/downloaded.
- pocketstar 12y ago>There's no one thing that is sure to keep your data around for decades. Using paper will keep your data around for decades.
- ggreer 12y agoThere are many ways in which paper can be destroyed. My mother has no pictures of her childhood because her home was burned down in a race riot. And analog media has many disadvantages. Storing, reproducing, transporting, searching, or modifying analog stuff is really hard.
- neals 12y agoCarbonite.com I have 1263776 files or 582GB backed-up for only $60,00 http://imgur.com/GiSjWkn http://imgur.com/GiSjWkn All servers, laptops and desktops I run occasionally put their files onto my main-desktop disc and everything gets uploaded to carbonite.
- alok-g 12y agoPer the website FAQs, it seems the $60 plan includes only data from "Users" folder and specific file types. Is there a way to back up other locations, and all file types? I have all my data in a separate partition, reaching 1.5 TB. With say 2 MBPS upload speed, it seems it would take about three months for the first upload. Also, did you face any issues with data limits from your ISP for the first upload?
- neals 12y agoIt might be because I have a 5 year old subscription and that other rules apply, but I pay the $60,00 and backup everything from .txt, .php, .cpp, many gigs of .avi etc. (I would never get to 500gb otherwise) No trouble from my ISP. Being from the Netherlands, I obviously have a 100/100 fiber connection. So it really isn't that much.
- quaffapint 12y agoI want to keep everything local. So I backup my important stuff to a usb drive that is inside a waterproof/fireproof case with a usb connector designed just for such a thing.
- rikkus 12y agoYou haven't heard good things about Crashplan? I have heard of one incident (reported on the web) where someone's data was lost at Crashplan HQ. But no system is infallible. It's generally excellent. It Just Works, it's cheap, fast, unobtrusive, flexible... My rules whenever anyone asks about backup: 1. Use backup systems, not file storage. Copying files somewhere is not a backup system. You could delete or corrupt files and never notice. Backup systems keep historic copies of files. 2. Use at least three backup locations. For example: Local hard drive, machine in a different building, the cloud. CrashPlan lets you do this, for example. 3. Test your backups regularly. It's boring, tedious and tricky, but it's less painful than losing your photos.
- dennisgorelik 12y agoIs losing your photos really THAT painful? The best photos are usually backed up on Facebook/etc. anyway.
- pfg 12y agoI keep all important files on Google Drive. Drive is syncing with my NAS, which is pushing a backup off-site to Amazon Glacier daily. Using TimeMachine on my laptop as well (for convenience).
- heyalexej 12y agoI have a very simple folder structure in my /home/ directory. ├── Code ├── Dirty ├── Downloads ├── Media │ ├── Books │ ├── Music │ ├── Pictures │ └── Videos ├── Projects └── Scripts I try to keep it very tidy because otherwise I would create a mess. "Dirty" and "Downloads" are working like /tmp, except they're getting wiped every 5 days [1]. Everything except "Downloads" and "Dirty" is backed up daily, incrementally and encrypted with s3cmd [2]. Some Inspiration here [3]. Sometimes I delete old stuff manually with s3cmd or from the S3 backend. Dotfiles and bootstrap scripts are stored on Github and Dropbox, so are serious projects. I also keep track of installed packages to be able to restore state on a new machine [4]. I'm not very emotional about restoring my machine. With a fast internet connection it takes me like 20-30 minutes to completely restore a wiped machine. [1] http://askubuntu.com/a/20831 http://askubuntu.com/a/20831 [2] http://s3tools.org/ http://s3tools.org/ [3] http://www.triatechnology.com/encrypted-incremental-backups-to-amazon-glacier-in-linux/ http://www.triatechnology.com/encrypted-incremental-backups-... [4] http://askubuntu.com/a/137991 http://askubuntu.com/a/137991
- unicornporn 12y agoFor every storage drive I have two backup drives. One of them I store in my home (but not connected to my computer) and one of them is stored encrypted (FileVault full disk encryption) at my friends place. I use this to backup: rsync -vaxXES --delete --backup --backup-dir="/Volumes/backup/__rsync/$(date +"%Y-%m-%d_%H%M%S")/" --log-file="/Volumes/backup/__rsync/$(date +"%Y-%m-%d_%H%M%S").log" --exclude="__rsync/" --exclude=".Trash/" --exclude=".Trashes/" --exclude=".Spotlight-V100" --exclude=".fseventsd/" --exclude=".DocumentRevisions-V100/" --exclude=".TemporaryItems/" /Volumes/data/ /Volumes/backup/ Sometimes i delete stuff in the __rsync dir manually. I've tried Crashplan and Backblaze, but they're crazy slow from Sweden.
- robinhoodexe 12y agoTime Machine for main backup (everything) Dropbox for documents (also nice to be accesible on my iPhone) Bitbucket (for dotfiles)
- nrr 12y agoI live laptop-only, and I buy my disks in triples. I do this even with SSDs because they, despite being solid state, are similarly flawed just like their spinning rust kin. Instead, their exhibited failure modes are less electromechanical and more simply "the universe is doing silly things to my data." (See also: solar flares, etc.) The other two disks get crammed into USB enclosures, and I use rsync to copy data around in a manner very similar to jwz's http://www.jwz.org/doc/backups.html http://www.jwz.org/doc/backups.html. With USB 3.0, this is very fast. That keeps me from experiencing a catastrophic data loss scenario, which is step one. It also solves a work continuity problem wherein I would otherwise be left now with the problem restoring from backups. Here, restoration is literally a matter of swapping disks and rebooting. Do I actually lose data? Well, yeah, but I've put an upper bound on it that is livable: I lose at most a week because the disk in my drawer at work gets swapped out every Friday for the one I keep at home. Some caveats: I don't use encryption at the level of my OS's VFS layer (yet), and I don't use RAID. I also don't use non-standard configurations like HFS+ with case sensitivity, btrfs, ZFS, etc. I keep most sensitive data (think private keys) elsewhere. Those are backed up by way of dumping to hardcopy and storing with my tax information. I can OCR the ASCII-armored representations later if I need to, and it makes choosing a fire safe easy since I don't need one that's rated for digital data storage. Now, this unfortunately doesn't solve the problem of storing historical copies of things, and I'm trying to find a way to accomplish that sanely without having to resort to alien technology like Plan 9's venti and fossil or vendor lock-in à la Time Machine, or rsyncing to dated directories.
- thaumaturgy 12y agoBackups are one of our specialties. First off, if you're in the market for a data recovery on your Drobo, I strongly recommend getting in touch with Gillware Data Recovery: https://gillware.com/ https://gillware.com/. Their prices are very fair, they have great staff, and they have a system that lets you preview your recovered data before paying anything. We got a new business client entirely because his RAID 1 NAS pooped itself and Gillware was able to recover everything he needed from it. If you've got a lot of data, online backup services are usually not an option. My usual advice then is to have a completely separate device that automatically pulls data from your primary device and stores backups in an archival format. If you have the time and inclination to do the setup on it, I really recommend BackupPC for this: http://backuppc.sourceforge.net/ http://backuppc.sourceforge.net/. I've used BackupPC for years for larger-scale backups; it's very reliable, it's not too difficult to get going, it does deduplication and compression and pooling and uses rsync (when possible) to reduce network load, it can easily be configured to send you an email in the event of trouble, and it has a simple web interface you can check into on a regular basis. I've yet to find a commercial shrink-wrapped product that does what BackupPC does as well as BackupPC does it. The downside of course is that you still wouldn't have offsite backups, so in the event of a fire or other local disaster, you might still be looking at complete data loss. If at all possible, try to pick a few of the most important directories of files, and get those backed up online. I haven't heard so many bad things about Crashplan or Backblaze, so I'm curious what you've heard.
- ricardobeat 12y agoI recently found myself with most of my data - accumulated over a 10-year period - in a single drive, and decided it was time to do something about it. My takeaways: - Avoid Backblaze unless you have your external drives connected at all times, files are erased after 30 days if not seen by the app. - GDrive is the cheapest, but can't backup external HDDs at all. - Arq/Glacier or Tarsnap is probably the safest archival you can get, but can be quite expensive depending on your usage pattern. - Crashplan, as Backblaze, focus on 'devices'. They are best suited to people who don't have lots of files outside of their computer. Crashplan has a terrible reputation for losing customer's data. In the end I went with Bitcasa[1] and am quite happy so far. It works the way I want it to: not necessary to mirror all data locally, can throw files adhoc at it, is accessible anywhere, decent web UI, mobile apps. Price is fair (€8/month for 1TB) and speeds are decent. It's the best I could find, still keeping my HDD backup in a closet just in case... [1] http://l.bitcasa.com/XFLTXjG7 http://l.bitcasa.com/XFLTXjG7 (with my referral code)
- izacus 12y agoI use Crashplan with two destinations: their cloud and Crashplan instance on my home NAS. This mitigates fire risk and keeps restores fast since I can just restore data from NAS instead of waiting for full restore from their cloud.