9 ms·

How a bug in Dropbox permanently deleted my 8000 photos

206 points by jancurn 12y ago

simi_ 12y ago
When your main business is storing files (and you also just raised yet another shitload of cash), not offering better safeguards against file loss is inadmissible. I left Dropbox for Google Drive when they announced the Condolezza Rice move, and I haven't looked back since.
the4aces 12y ago
Try going into the folders and manually restoring. You can select more than one file at a time.... I was able to restore for more than 30 days... Worth a shot.
- jancurn 12y ago
  Many thanks for your advice, this was the first thing I tried, unfortunately without any success...
CaptainZapp 12y ago
I use Dropbox for sharing pictures, but would never even dream to use a cloud based service for backup purposes. Granted, Jan's case is a bit more complex and I'm really sorry for his loss. Stories like that should really be a lesson to everybody never to completely trust a cloud based service as your main backup. On a side note: I agree that archiving of digital files is a hard problem. The smartest librarians of the world are thinking about how to achieve this for, literally, decades and I'm not sure they even have a good solution to the problem. My personal strategy is redundancy: I buy new hard disks every couple of years and copy all important files, twice. One hard disk is kept off site. It's not perfect, but it's the best I can come up with. Reading horror stories, like Jan's, indicates that it's the better solution. Despite the messiness.
- blablabla123 12y ago
  I've bought myself a Blue Ray recorder half a year ago and it's as easy to use as Dropbox with OS X...
  - mkesper 12y ago
    Are you using dvdisaster to secure against data-loss?
    - slantyyz 12y ago
      I would also add that when using optical discs for backup, M-Discs are probably preferable over conventional dye-based discs.
      - blablabla123 12y ago
        Wow, do they work with conventional recorders?
        dublinben 12y ago
        No, but they work with conventional readers. You need a special laser to do the physical etching that they're doing.
        slantyyz 12y ago
        Only if they say "M-Disc" on it. Many consumer recorders, especially if made by LG, support writing to these discs.
    - blablabla123 12y ago
      Thanks for the hint, no, but I'll do that in the future. In the past I actually had the problem of errorneous blocks one or two times. Eventhough these were Noname Multisession CDs that lay too much in the sun. (The media actually turned yellowish). Never had this problem again, but I only use brand media which I put it into proper covers that don't get too much light. But I'll use it in the future... (Not sure if that's of interest anymore, but I used TrueCrypt for business backups.)
mahouse 12y ago
Well, sorry to say this, but if all of your photos were stored only online...
- morganvachon 12y ago
  It depends on how you look at it. They were stored online (Dropbox servers) as well as locally (Macbook). The problem is that a bug in Dropbox's client turned a de-sync operation into a delete operation. So one way to see it is that even having local storage was not safe because the sync was flawed. Another possible way to look at it is if your only local copy is synced with the cloud, then effectively your only storage is online.
petercooper 12y ago
If you are using Dropbox as a sole backup of your files, think again. Simplify to If you only have a sole backup, think again. Points 4 and 5 should be you always have more than one backup of important material and at least one backup should be on physical media that you own.
- zedadex 12y ago
  >> If you are using Dropbox as a sole backup of your files, think again. > Simplify to If you only have a sole backup, think again. Yep. 3-2-1: "At least three copies, in two different formats, with one of those copies off-site."
  - sudorank 12y ago
    I like keeping backups on a hard disk in the office. For a guy who grew up in the 90's there's something comforting about being able to physically look at one of your backup methods!
    - dublinben 12y ago
      That's a great backup method, which is cheap and easy to recover. It's not very robust though. In the event of a natural disaster, your office is just as likely to be destroyed as your home. Ideally you would have an offsite backup (or two) in a completely different region.
      - pyre 12y ago
        The 'different region' requirement might just depend on different things: - How far away are your office and home? - What is your locality? As an example, I don't live in a place prone to earthquakes or hurricanes, and my office is a 20 ~ 30 min drive from my home. What is the risk that both my office and home are destroyed?
        dublinben 12y ago
aaronem 12y ago
> If you are using Dropbox as a sole backup of your files, think again. Good advice. > Without making a mistake, you might lose your files. If Dropbox, or any other single service or method, constitutes your entire backup strategy, then the mistake is already made.
- RoadRunner_23 12y ago
  I agree that single service or method should not be entire backup strategy. I backup pictures and important files to two seperate external discs, besides desktop's storage. Sorry to hear, but sometimes it happens. Look at the positive side. you will have more precious moments to look forward to and those lost pictures, you will have to go old fashioned way of recollecting. Do you remember the day...kind.
profsnuggles 12y ago
It really sucks that we all have to keep learning this lesson over and over. Everyone I've ever spoken to that has a good backup strategy in place has it because they have lost irreplaceable files. Have real backups. Syncing is not a backup strategy, raid is not a backup strategy, etc. 3-2-1 At least three copies in 2 different (storage) formats and at least one copy offsite. It sounds like overkill but you have to decide how much your data is really worth.
- catshirt 12y ago
  i think people are still struggling because creating real backups still seems like kind of a pain in the ass compared to cloud syncing. what's currently the most convenient and simple way to store multiple local backups?
  - profsnuggles 12y ago
    It IS a pain and I don't know that there is a way to fix that. As I said though you have to decide what your data is worth to you. I personally use http://duplicity.nongnu.org/ http://duplicity.nongnu.org/ and a semi-complicated system of cron and bash script to backup my files to my local server, some flash drives and s3.
  - slantyyz 12y ago
    Getting a Time Capsule is pretty convenient if you're a Mac user. That you can add USB drives to the Time Capsule also allows you to store extra discrete snapshots using other backup software like Crashplan.
- arethuza 12y ago
  "raid is not a backup strategy" Though I have seen it used (to my horror) as a software release strategy.... :-)
SunboX 12y ago
This happened to me (in German): http://www.basicthinking.de/blog/2012/04/18/fur-immer-ausgesperrt-bei-google-eine-leidensgeschichte/ http://www.basicthinking.de/blog/2012/04/18/fur-immer-ausges...
- jabiko 12y ago
  The linked article is about someone who managed to lock himself out of his Google account. No word about Dropbox.
gabemart 12y ago
I think this bug should be considered completely separately from how unwise it is to use a cloud service as the sole storage of important files. Regardless of the circumstances, losing user files against the wishes of the user is the absolute worst thing a cloud backup provider can do. Even for files that are deleted intentionally and unambiguously by the user, I'm astonished that Dropbox actually deletes the files at the end of the 30-day restore window. I would expect them to keep the files for some multiple of the publicly-stated restore deadline where the multiple >= 2, if for no other reason than as a goodwill generator. There is no more evangelical advocate for your company than the customer you email to say "Yes, you intentionally deleted this file six weeks ago. The 30 day deletion deadline has passed, but I have managed to restore the most recent version of your dissertation. Thank you for using Dropbox." For files that aren't intentionally deleted by the user but are "de-synced", it is disgraceful and appalling that there is no contingency system in place. Keeping user files when the user assumes or wishes them to be kept safe is the core competency of a service like Dropbox. "The user should have kept multiple redundant copies" is not an excuse for a poorly managed online backup service. "Keep multiple backups of everything important" is good advice for a user, but "Keep user files safe when the user thinks they are safe" is the most essential advice imaginable for the CEO of an online backup service.
- icebraining 12y ago
  I would expect them to keep the files for some multiple of the publicly-stated restore deadline where the multiple >= 2, if for no other reason than as a goodwill generator. On the other hand, keeping copies of the user's data after you say you've deleted it doesn't sound too ethical to me. I don't expect immediate purging, but I also don't expect potentially sensitive files to linger for months after they were supposed to be deleted. I think a better system would be to warn about big deletion events (e.g. send an email) to allow the user to revert it within the 30-day period.
  - gabemart 12y ago
    > On the other hand, keeping copies of the user's data after you say you've deleted it doesn't sound too ethical to me. I don't expect immediate purging, but I also don't expect potentially sensitive files to linger for months after they were supposed to be deleted. This is a fair concern, but I think the issue of recovering files is much more important than the issue of retention of deleted files to the huge majority of the Dropbox user base. I imagine most privacy-conscious users would use an online sync service that offers encryption by default, etc.
uladzislau 12y ago
Dropbox is NOT a backup service. It's a convenient file syncing and sharing service. And even then it's a good ides to actually check docs or help before proceeding to do something which might result In a loss of data.
codva 12y ago
I think the lesson here is not that the cloud is bad, it's that a sync service is not a backup. I use Dropbox to share, I use Amazon S3 to backup.
egypturnash 12y ago
Sync is not backup. Sync is not backup. Sync is not backup. My strategy: a big external drive used for Time Machine, and a subscription to Backblaze. Both of these are all about retaining multiple versions, recovering from accidental deletion, and continuously backing up in the background. Dropbox is about syncing stuff between computers.
- bruceboughton 12y ago
  I too am a Backblaze customer but beware: backups of deleted files are deleted after 30 days.
  - TheLoneWolfling 12y ago
    Your definition of backup...
    - bruceboughton 12y ago
      Which is why I don't solely use Backblaze (fast recovery, medium level of trust). I also backup the same data to Glacier-class S3 (slow recovery, high level of trust) and Time Machine (v. quick recovery, same-site risk).
  - lh7777 12y ago
    One of the reasons I switched to Crashplan. Their Java client is pretty terrible performance-wise, but it's still the best cloud backup I've tried.
  - smackfu 12y ago
    That's kind of surprising given how their hardware model is setup, with pods of HDDs that are filled up and then essentially archived away. I guess the amount of space freed up is enough that it makes sense to open those pods back up for writing.
  - egypturnash 12y ago
    Huh, I didn't know that. My first line of 'oh shit I deleted a thing' is Time Machine, since it's right there next to my external monitor and won't require waiting for a download.
restlessmedia 12y ago
We live in a convenient age, but also a fickle one. I laugh when my wife prints 100 hundreds of digital photos although her disaster recovery process is more stable than mine.
- nly 12y ago
  100 hundreds is a lot printing ;)
- morganvachon 12y ago
  My wife does the same; she has a Snapfish account. She will have them print and mail her the most important pictures from a set, while trusting the ones she doesn't care as much about to our home server and Facebook (the former for long-term storage, the latter for convenient access). Since she has to order a certain number of prints every year to keep the account open, and it comes with unlimited free storage, it works out well for her. The only exception was our wedding photos. We have a set of prints directly from the studio hired to take the pictures, as well as a digital copy on DVD from them. We also have those digital files backed up on our home server, a computer at my parents' house, Dropbox, Snapfish, and both of our Facebook accounts. The photography company also has long-term storage that we would have access to if all else failed. That's about as redundant as we can get without spending a ton of money, but it's more than most people seem to have.
- ToastyMallows 12y ago
  I laughed at this when I first read it, but it's seriously true. For people that aren't computer literate, physical copies can sometimes be the best form of a backup for digital data.
Aardwolf 12y ago
I don't trust anything called "sync". I trust file operations "move", "copy" and "remove". But "sync" could mean anything: which side is syncing to which side? If something is not present on one side, will it delete it on the other? I wish the term didn't exist, and apps/clients/... would copy or even just load or show things, and if you want to "sync", then rsync with clear source and destination. Same with mail for example: I don't want to "sync" email messages, I just want to see them, the most recent ones first.
- icebraining 12y ago
  Here "sync" means what Dropbox says it means. The answers to your questions are well defined in their software, even if to some other software, they might be different. I wish the term didn't exist, and apps/clients/... would copy or even just load or show things, and if you want to "sync", then rsync with clear source and destination. The source and destination is always clear. It's to the server and from the server. Dropbox uses a star topology, not P2P. You'd get the same result if you used rsync with source and destination, if you passed --delete, which is equivalent to what Dropbox does.
  - deleted 12y ago
    [deleted]
  - Aardwolf 12y ago
    "The source and destination is always clear. It's to the server and from the server." That's exactly what I mean by not clear: to the server and from the server are the exact opposites. So if something is not present on one side, it can do two things to make both sides equal: it can delete it from the one side, or it can copy it to the other side. If all it has is a "sync" button, not a "sync to server" or "sync from server" button, it's not clear what it will do. Something that deletes files shouldn't be called sync, but should clearly be marked as a delete operation. And deleting something from one location should not have the surprise effect of deleting it on other locations too, unless it's called "remove everywhere".
    - 12y ago
shimshim 12y ago
Just curious why more people don't learn about and invest in home NAS's? Something like Synology would be pretty good at storing 8000 important photos. It is still a single point of failure, but its one I control.
- fulafel 12y ago
  It's a hassle to learn, admin, and it's more likely to eat your data. Many people do not have real internet connections at home so they could assign it an ip and access it remotely. Plus it's hard to buy a good one unless you go for generous overkill. And after 2 years you have yet another vendor abandoned Linux device with remote vulnerabilities on your home net. (I still prefer it to cloud storage though)
  - shimshim 12y ago
    Thanks for the point of view. I personally love the Synology at home and have a spare disk for cases where I may have a failure. I would say, barring something catastrophic, that I am set for another 3-5 years, which is fine. I would most likely buy a new model Synology 3 years from now and continue expansion. They also offer another disk shelf you can addon.
yoda_sl 12y ago
As many others mentioned already "sync is not backup". If you really want true backup then on a Mac I will suggest to have a setup like: 1- local backup for quick recovery with Time Machine 2- remote backup with a true backup solution like CrashPlan (or similar) I personally have such setup with a local one with my Time Machine, and having CrashPlan running all the time, it did save me a couple times when after many weeks I deleted a wrong file or folder... Got everything back without problem. If you don't have a Time Capsule, then you can always consider an alternative with just a smaller subset of what you want to archive by using a JetDrive by Transcend. I recently got a 128GB JetDrive and it is my local destination for my MBP for TimeMachine... Yes 128GB is not enough but knowing that CrashPlan is constantly running make me comfortable enough with that setup.
- slantyyz 12y ago
  I use my Time Capsule's USB port to attach an external drive that is shared so that Crashplan can backup to a folder on it. That's in addition to using Crashplan Central for cloud backups and Time Machine.
pierreio 12y ago
> "you should backup" What about in 10, 20 years? Photo libraries will keep inflating. Local storage will not. As of now I backup from a SSD Mac. What happens when I don't have a computer anymore? Interestingly, people don't value "bits" or information. We value moments and emotions and work and art. There's no successful current consumer business model for people to store and backup photos (Backblaze is mainly prosumer). And so aren't social networks the real backups by now? The redundancy of publishing on multiple services means some photos will stick and the rest will fade, somehow like former printed pictures I guess. Publish it or lose it?
- walterbell 12y ago
  > Local storage will not. External backup drives continue to increase in size at a falling cost per GB. It is cheaper and lower latency to store/retrieve a large amount of data locally than remotely, unless you believe that the current glut of "near-free cloud storage" will continue indefinitely. Market cycles suggest otherwise. > Publish it or lose it Why do social networks store your photos "for free"?
- icebraining 12y ago
  What happens when I don't have a computer anymore? Wifi-enabled drives are already common; no computer needed. They are even targeted at mobile device users ("for iPad/iPhone").
- artmageddon 12y ago
  > And so aren't social networks the real backups by now? I hope not. I cringe when I see people who treat them like they are. What do you do when your account password gets cracked(it happens to people on Facebook a lot), or the site itself goes away(MySpace)? Also, you're not able to store the images in their original form; I'm pretty sure sites like these will have limitations on what you can upload. I'm not a pro-photographer by any means, but there's no way I could upload a RAW file to Facebook.
rcthompson 12y ago
I would never consider using Dropbox (or any other cloud storage) as my sole backup system for critical files unless I was willing to pay for the "Packrat", i.e. infinite history, feature.
rayiner 12y ago
The bug in question: > From all this information it seems that Dropbox client first deletes files locally before it informs the server about the new selective sync settings. Consequently, if the client crashes or is killed before the server is contacted, the files remain deleted without any trace. After the client restarts again, it only sees there are some files missing and syncs this new state with the server. It concerns me that the top several comments are some variation of "you shouldn't trust the cloud with your data." How did software services become so totally insulated from the expectations we have of every other service that holds on to our valuable property?
- kalleboo 12y ago
  > It concerns me that the top several comments are some variation of "you shouldn't trust the cloud with your data." No, it's "you shouldn't trust one single thing with your data". Hardware fails. OSes have bugs (OS X's "space in your drive name and we delete your user folder.") Services have unexpected gotchas. This is why you never store anything in only one place. A sync service by definition is not a separate place (unless he had found a bug in the Packrat feature)
  - rayiner 12y ago
    My point is that, for some reason, we have been conditioned to believe that bugs and data loss are just something that happen as opposed to something that service providers, who we pay money to, are charged with avoiding. We hold software to a much lower standard of quality than we hold other products and services.
    - kalleboo 12y ago
      * we have been conditioned to believe that bugs and data loss are just something that happen They're something that happen to unregulated services that cost $8/month. If I paid $8/month for electricity, water or Internet access, yeah I'd expect yearly blackouts. To get an SLA you have to pay more.
      - 12y ago
johnnyladders 12y ago
If it's not saved in three places, it's not saved.
zperrault 12y ago
I noticed there hasn't been any mention of attempting to recover your deleted files from your local disk. If you haven't tried it yet it is worth a shot. There are a few software solutions out there that will scan your disk for files that have been deleted. On Windows I have used Piriform's Recuva[1] and on OS X I have used PhotoRec[2], both have worked rather well. It is worth noting that the longer you wait to try the more likely the sectors are to be overwritten with new data. (And you should back up any files currently on the disk). I know it is a long shot at this point but some chance is better than none. Good luck. [1] http://www.piriform.com/recuva http://www.piriform.com/recuva [2] http://www.cgsecurity.org/wiki/PhotoRec http://www.cgsecurity.org/wiki/PhotoRec
- dpcan 12y ago
  Or, I wouldn't be surprised if tomorrow he posts that he found them all in his trash.... Because nobody ever thinks to clear it. Sometimes the simplest solution is right under our nose.
  - pyre 12y ago
    He said that he de-synced them to free up space. I'm assuming that he would have noticed if the operation didn't free up space.
JoeAltmaier 12y ago
My buddy Tom says "If its not on your current computer, its lost". Imagine when Dropbox closes one day (as every company ultimately does), how much will be lost.
- therealmocker 12y ago
  Since all the files are sync'ed to the computer, nothing would be lost unless selective sync was enabled.
  - rockdoe 12y ago
    A comparable bug would be that when the server goes away (Dropbox closes), the client starts deleting all local files.
steven2012 12y ago
In today's digital age, I believe lots and lots of people will lose a lot of valuable information like photos because they don't have a good understanding of what backing up means. I've warned my friend who is computer illiterate that he is risking losing his childrens' complete photo collection for their entire lives because he is only keeping a single copy of his data on his hard drive. This is probably the situation with millions of people in the US alone, and when that hard drive failure event happens, which will happen, then they will lose everything.
- DanBC 12y ago
  Did that work? Would it have been more effective to just provide him with a list of stuff - a choice of two drives; a choice of two softwares (with really easy URLs to buy and download) and instructions for using and testing. That could be a gift for birthday or Xmas?
  - steven2012 12y ago
    I bought him a new hard drive, copied the photos over and gave him instructions on how to do it. He hasn't bothered to update his backup once, and it's been about 5 years and his photo collection has only grown.
    - kngspook 12y ago
      I'm thinking of buying a CrashPlan Business account (my corp uses the Enterprise version) for my family for this reason. I set them up with TimeMachine, but even plugging in a USB drive has proven to be too much to remember.
- ominous_prime 12y ago
  This is becoming even more important with SSDs. I used to do a lot of data recovery, and photos were of course a very common type of data that customers wanted to recover. If the drive was spinning, I could usually recover most of the data. Even if the drive was completely dead, there was still some chance at recovery of I had the correct logic board, and for a hefty price I could send it out to a company with more resources to get the platters spinning again. SSDs can be much harder recover from, since you need information stored in the controller to map out the data, and many consumer level hard drives brick themselves on failure to prevent silent data corruption.
servowire 12y ago
The solution to this kind of problems is: Make Read-only "disaster recovery" backups - zip it and burn it to a DVD set or Bluray, and put it off-site. Make incrementals or differentials of those discs every few months and a full data set each 3 months. Takes a few hours, I like to burn all my data to Bluray (50GB per disk) while watching a movie. Store the bluray data sets at a friend's house and never worry again.
filmgirlcw 12y ago
I had an issue similar to this (but fortunately not nearly as bad) about two and a half or three years ago and that was what convinced me to pay of the Packrat service add-on. I agree with the OP that that should be a built-in option for all paying customers -- or at least make it more visible as an add-on. I've had instances in the past where it was months later that I realized something was either deleted or didn't sync and I had to look through Events and use Packrat to restore. It can be scary - even if you do make backups of your backups.
danbee 12y ago
The moment you only have one copy of your files anywhere you don't have a backup. I've heard it said that if a file doesn't exist in at least three different places it might as well not exist.
- kngspook 12y ago
  Exactly. (I can't believe I had to scroll down this far to find this.)
vondur 12y ago
I worry about stuff like that happening to me with my photos. The best idea I've come up with is setting up a Synology that I export my photos to. It has a basic web app for showing off photos. I then use it's AWS S3 service to back those photos up every night. The Synology also has something like RAID1 on it for some redundancy at the local level. The hard part is remembering to export the files from the computer to the Synology.
mhartl 12y ago
Thanks for the tip on the Packrat feature. I didn't know about that. Added.
je42 12y ago
This is clearly a problem for the reputation of dropbox. On their website they market the service as: "Even if your phone goes for a swim, your stuff is always safe in Dropbox and can be restored in a snap." With this message in mind, and a valuation of roughly >>10$ BILLION<<. I think, you can have very very high expectations regarding the data retention of their service. Also, the whole point of using a service like dropbox is to remove friction/time investment regarding the standard backup/sync tasks. If you suggests to handle multiple harddisk backups locally+offsite, that is fine except that not everyone wants this kind of time investment and cost associated with a do-it-yourself backup service and rather depend on a service provider that they can trust.
wrs 12y ago
As I said to someone the other day after dealing with yet another failure by Dropbox to do the safe thing or even the reasonably-recoverable thing with our data: "One reason I'm not worth a billion dollars is I would never have let Dropbox ship like this!" Doing sync on top of an existing filesystem, making it work with existing apps, maintaining solid transaction semantics, and having a simple, understandable user model -- you just can't do all four. The Dropbox experiment has now shown that "correctness" doesn't matter, commercially speaking.
PhantomGremlin 12y ago
Dropbox has one of the worst websites on the Internet. Seriously! E.g. people have mentioned that "packrat" would have helped in this situation. But try finding that service from the pricing page. Hint: NOT THERE. Someone here linked to some explanation in their "help" section. Seriously? Such an important feature is only mentioned under "help"? Even if you go to sign up for the paid version, Dropbox Pro, packrat isn't mentioned [1]. Their horrid website alone is enough for me to not want to do business with them. Someone else here has succinctly summarized the situation [2]: One reason I'm not worth a billion dollars is I would never have let Dropbox ship like this! [1] https://www.dropbox.com/upgrade https://www.dropbox.com/upgrade [2] https://news.ycombinator.com/item?id=8105548 https://news.ycombinator.com/item?id=8105548
Fofer 12y ago
Once you deleted your own local copy, the folder of photos on Dropbox was no longer a "backup." You just moved your one copy elsewhere. You essentially had NO backup. And everyone should know, you should always have at least two backups. (Again, you had zero.) I wouldn't even consider Dropbox (without Packrat) a valid backup destination either, because any changes get synced. Sorry this happened to you, but better backup practices could have avoided it easily.
ecnahc515 12y ago
I use both backblaze, as well as Google Drive. I don't have anything in particular I'd be completely upset if I lost it forever, which is why I dont do local backups, but I've definitely been looking at using Crashplan as a secondary backup. I just cant be bothered to hook my laptop up to an external when I hardly sit next to it anyway. What does HN think? Safe if I went with 2 cloud backup providers?

206 points by jancurn 12y ago

simi_ 12y ago

When your main business is storing files (and you also just raised yet another shitload of cash), not offering better safeguards against file loss is inadmissible. I left Dropbox for Google Drive when they announced the Condolezza Rice move, and I haven't looked back since.

the4aces 12y ago

Try going into the folders and manually restoring. You can select more than one file at a time.... I was able to restore for more than 30 days... Worth a shot.

jancurn 12y ago
Many thanks for your advice, this was the first thing I tried, unfortunately without any success...

CaptainZapp 12y ago

I use Dropbox for sharing pictures, but would never even dream to use a cloud based service for backup purposes. Granted, Jan's case is a bit more complex and I'm really sorry for his loss. Stories like that should really be a lesson to everybody never to completely trust a cloud based service as your main backup. On a side note: I agree that archiving of digital files is a hard problem. The smartest librarians of the world are thinking about how to achieve this for, literally, decades and I'm not sure they even have a good solution to the problem. My personal strategy is redundancy: I buy new hard disks every couple of years and copy all important files, twice. One hard disk is kept off site. It's not perfect, but it's the best I can come up with. Reading horror stories, like Jan's, indicates that it's the better solution. Despite the messiness.

blablabla123 12y ago
I've bought myself a Blue Ray recorder half a year ago and it's as easy to use as Dropbox with OS X...
- mkesper 12y ago
  Are you using dvdisaster to secure against data-loss?
  - slantyyz 12y ago
    I would also add that when using optical discs for backup, M-Discs are probably preferable over conventional dye-based discs.
    - blablabla123 12y ago
      Wow, do they work with conventional recorders?
      - dublinben 12y ago
        No, but they work with conventional readers. You need a special laser to do the physical etching that they're doing.
      - slantyyz 12y ago
        Only if they say "M-Disc" on it. Many consumer recorders, especially if made by LG, support writing to these discs.
  - blablabla123 12y ago
    Thanks for the hint, no, but I'll do that in the future. In the past I actually had the problem of errorneous blocks one or two times. Eventhough these were Noname Multisession CDs that lay too much in the sun. (The media actually turned yellowish). Never had this problem again, but I only use brand media which I put it into proper covers that don't get too much light. But I'll use it in the future... (Not sure if that's of interest anymore, but I used TrueCrypt for business backups.)

mahouse 12y ago

Well, sorry to say this, but if all of your photos were stored only online...

morganvachon 12y ago
It depends on how you look at it. They were stored online (Dropbox servers) as well as locally (Macbook). The problem is that a bug in Dropbox's client turned a de-sync operation into a delete operation. So one way to see it is that even having local storage was not safe because the sync was flawed. Another possible way to look at it is if your only local copy is synced with the cloud, then effectively your only storage is online.

petercooper 12y ago

If you are using Dropbox as a sole backup of your files, think again. Simplify to If you only have a sole backup, think again. Points 4 and 5 should be you always have more than one backup of important material and at least one backup should be on physical media that you own.

zedadex 12y ago
>> If you are using Dropbox as a sole backup of your files, think again. > Simplify to If you only have a sole backup, think again. Yep. 3-2-1: "At least three copies, in two different formats, with one of those copies off-site."
- sudorank 12y ago
  I like keeping backups on a hard disk in the office. For a guy who grew up in the 90's there's something comforting about being able to physically look at one of your backup methods!
  - dublinben 12y ago
    That's a great backup method, which is cheap and easy to recover. It's not very robust though. In the event of a natural disaster, your office is just as likely to be destroyed as your home. Ideally you would have an offsite backup (or two) in a completely different region.
    - pyre 12y ago
      The 'different region' requirement might just depend on different things: - How far away are your office and home? - What is your locality? As an example, I don't live in a place prone to earthquakes or hurricanes, and my office is a 20 ~ 30 min drive from my home. What is the risk that both my office and home are destroyed?
      - dublinben 12y ago

aaronem 12y ago

> If you are using Dropbox as a sole backup of your files, think again. Good advice. > Without making a mistake, you might lose your files. If Dropbox, or any other single service or method, constitutes your entire backup strategy, then the mistake is already made.

RoadRunner_23 12y ago
I agree that single service or method should not be entire backup strategy. I backup pictures and important files to two seperate external discs, besides desktop's storage. Sorry to hear, but sometimes it happens. Look at the positive side. you will have more precious moments to look forward to and those lost pictures, you will have to go old fashioned way of recollecting. Do you remember the day...kind.

profsnuggles 12y ago

It really sucks that we all have to keep learning this lesson over and over. Everyone I've ever spoken to that has a good backup strategy in place has it because they have lost irreplaceable files. Have real backups. Syncing is not a backup strategy, raid is not a backup strategy, etc. 3-2-1 At least three copies in 2 different (storage) formats and at least one copy offsite. It sounds like overkill but you have to decide how much your data is really worth.

catshirt 12y ago
i think people are still struggling because creating real backups still seems like kind of a pain in the ass compared to cloud syncing. what's currently the most convenient and simple way to store multiple local backups?
- profsnuggles 12y ago
  It IS a pain and I don't know that there is a way to fix that. As I said though you have to decide what your data is worth to you. I personally use http://duplicity.nongnu.org/ http://duplicity.nongnu.org/ and a semi-complicated system of cron and bash script to backup my files to my local server, some flash drives and s3.
- slantyyz 12y ago
  Getting a Time Capsule is pretty convenient if you're a Mac user. That you can add USB drives to the Time Capsule also allows you to store extra discrete snapshots using other backup software like Crashplan.
arethuza 12y ago
"raid is not a backup strategy" Though I have seen it used (to my horror) as a software release strategy.... :-)

SunboX 12y ago

This happened to me (in German): http://www.basicthinking.de/blog/2012/04/18/fur-immer-ausgesperrt-bei-google-eine-leidensgeschichte/ http://www.basicthinking.de/blog/2012/04/18/fur-immer-ausges...

jabiko 12y ago
The linked article is about someone who managed to lock himself out of his Google account. No word about Dropbox.

gabemart 12y ago

I think this bug should be considered completely separately from how unwise it is to use a cloud service as the sole storage of important files. Regardless of the circumstances, losing user files against the wishes of the user is the absolute worst thing a cloud backup provider can do. Even for files that are deleted intentionally and unambiguously by the user, I'm astonished that Dropbox actually deletes the files at the end of the 30-day restore window. I would expect them to keep the files for some multiple of the publicly-stated restore deadline where the multiple >= 2, if for no other reason than as a goodwill generator. There is no more evangelical advocate for your company than the customer you email to say "Yes, you intentionally deleted this file six weeks ago. The 30 day deletion deadline has passed, but I have managed to restore the most recent version of your dissertation. Thank you for using Dropbox." For files that aren't intentionally deleted by the user but are "de-synced", it is disgraceful and appalling that there is no contingency system in place. Keeping user files when the user assumes or wishes them to be kept safe is the core competency of a service like Dropbox. "The user should have kept multiple redundant copies" is not an excuse for a poorly managed online backup service. "Keep multiple backups of everything important" is good advice for a user, but "Keep user files safe when the user thinks they are safe" is the most essential advice imaginable for the CEO of an online backup service.

icebraining 12y ago
I would expect them to keep the files for some multiple of the publicly-stated restore deadline where the multiple >= 2, if for no other reason than as a goodwill generator. On the other hand, keeping copies of the user's data after you say you've deleted it doesn't sound too ethical to me. I don't expect immediate purging, but I also don't expect potentially sensitive files to linger for months after they were supposed to be deleted. I think a better system would be to warn about big deletion events (e.g. send an email) to allow the user to revert it within the 30-day period.
- gabemart 12y ago
  > On the other hand, keeping copies of the user's data after you say you've deleted it doesn't sound too ethical to me. I don't expect immediate purging, but I also don't expect potentially sensitive files to linger for months after they were supposed to be deleted. This is a fair concern, but I think the issue of recovering files is much more important than the issue of retention of deleted files to the huge majority of the Dropbox user base. I imagine most privacy-conscious users would use an online sync service that offers encryption by default, etc.

uladzislau 12y ago

Dropbox is NOT a backup service. It's a convenient file syncing and sharing service. And even then it's a good ides to actually check docs or help before proceeding to do something which might result In a loss of data.

codva 12y ago

I think the lesson here is not that the cloud is bad, it's that a sync service is not a backup. I use Dropbox to share, I use Amazon S3 to backup.

egypturnash 12y ago

Sync is not backup. Sync is not backup. Sync is not backup. My strategy: a big external drive used for Time Machine, and a subscription to Backblaze. Both of these are all about retaining multiple versions, recovering from accidental deletion, and continuously backing up in the background. Dropbox is about syncing stuff between computers.

bruceboughton 12y ago
I too am a Backblaze customer but beware: backups of deleted files are deleted after 30 days.
- TheLoneWolfling 12y ago
  Your definition of backup...
  - bruceboughton 12y ago
    Which is why I don't solely use Backblaze (fast recovery, medium level of trust). I also backup the same data to Glacier-class S3 (slow recovery, high level of trust) and Time Machine (v. quick recovery, same-site risk).
- lh7777 12y ago
  One of the reasons I switched to Crashplan. Their Java client is pretty terrible performance-wise, but it's still the best cloud backup I've tried.
- smackfu 12y ago
  That's kind of surprising given how their hardware model is setup, with pods of HDDs that are filled up and then essentially archived away. I guess the amount of space freed up is enough that it makes sense to open those pods back up for writing.
- egypturnash 12y ago
  Huh, I didn't know that. My first line of 'oh shit I deleted a thing' is Time Machine, since it's right there next to my external monitor and won't require waiting for a download.

restlessmedia 12y ago

We live in a convenient age, but also a fickle one. I laugh when my wife prints 100 hundreds of digital photos although her disaster recovery process is more stable than mine.

nly 12y ago
100 hundreds is a lot printing ;)
morganvachon 12y ago
My wife does the same; she has a Snapfish account. She will have them print and mail her the most important pictures from a set, while trusting the ones she doesn't care as much about to our home server and Facebook (the former for long-term storage, the latter for convenient access). Since she has to order a certain number of prints every year to keep the account open, and it comes with unlimited free storage, it works out well for her. The only exception was our wedding photos. We have a set of prints directly from the studio hired to take the pictures, as well as a digital copy on DVD from them. We also have those digital files backed up on our home server, a computer at my parents' house, Dropbox, Snapfish, and both of our Facebook accounts. The photography company also has long-term storage that we would have access to if all else failed. That's about as redundant as we can get without spending a ton of money, but it's more than most people seem to have.
ToastyMallows 12y ago
I laughed at this when I first read it, but it's seriously true. For people that aren't computer literate, physical copies can sometimes be the best form of a backup for digital data.

Aardwolf 12y ago

I don't trust anything called "sync". I trust file operations "move", "copy" and "remove". But "sync" could mean anything: which side is syncing to which side? If something is not present on one side, will it delete it on the other? I wish the term didn't exist, and apps/clients/... would copy or even just load or show things, and if you want to "sync", then rsync with clear source and destination. Same with mail for example: I don't want to "sync" email messages, I just want to see them, the most recent ones first.

icebraining 12y ago
Here "sync" means what Dropbox says it means. The answers to your questions are well defined in their software, even if to some other software, they might be different. I wish the term didn't exist, and apps/clients/... would copy or even just load or show things, and if you want to "sync", then rsync with clear source and destination. The source and destination is always clear. It's to the server and from the server. Dropbox uses a star topology, not P2P. You'd get the same result if you used rsync with source and destination, if you passed --delete, which is equivalent to what Dropbox does.
- deleted 12y ago
  [deleted]
- Aardwolf 12y ago
  "The source and destination is always clear. It's to the server and from the server." That's exactly what I mean by not clear: to the server and from the server are the exact opposites. So if something is not present on one side, it can do two things to make both sides equal: it can delete it from the one side, or it can copy it to the other side. If all it has is a "sync" button, not a "sync to server" or "sync from server" button, it's not clear what it will do. Something that deletes files shouldn't be called sync, but should clearly be marked as a delete operation. And deleting something from one location should not have the surprise effect of deleting it on other locations too, unless it's called "remove everywhere".
  - 12y ago

shimshim 12y ago

Just curious why more people don't learn about and invest in home NAS's? Something like Synology would be pretty good at storing 8000 important photos. It is still a single point of failure, but its one I control.

fulafel 12y ago
It's a hassle to learn, admin, and it's more likely to eat your data. Many people do not have real internet connections at home so they could assign it an ip and access it remotely. Plus it's hard to buy a good one unless you go for generous overkill. And after 2 years you have yet another vendor abandoned Linux device with remote vulnerabilities on your home net. (I still prefer it to cloud storage though)
- shimshim 12y ago
  Thanks for the point of view. I personally love the Synology at home and have a spare disk for cases where I may have a failure. I would say, barring something catastrophic, that I am set for another 3-5 years, which is fine. I would most likely buy a new model Synology 3 years from now and continue expansion. They also offer another disk shelf you can addon.

yoda_sl 12y ago

As many others mentioned already "sync is not backup". If you really want true backup then on a Mac I will suggest to have a setup like: 1- local backup for quick recovery with Time Machine 2- remote backup with a true backup solution like CrashPlan (or similar) I personally have such setup with a local one with my Time Machine, and having CrashPlan running all the time, it did save me a couple times when after many weeks I deleted a wrong file or folder... Got everything back without problem. If you don't have a Time Capsule, then you can always consider an alternative with just a smaller subset of what you want to archive by using a JetDrive by Transcend. I recently got a 128GB JetDrive and it is my local destination for my MBP for TimeMachine... Yes 128GB is not enough but knowing that CrashPlan is constantly running make me comfortable enough with that setup.

slantyyz 12y ago
I use my Time Capsule's USB port to attach an external drive that is shared so that Crashplan can backup to a folder on it. That's in addition to using Crashplan Central for cloud backups and Time Machine.

pierreio 12y ago

> "you should backup" What about in 10, 20 years? Photo libraries will keep inflating. Local storage will not. As of now I backup from a SSD Mac. What happens when I don't have a computer anymore? Interestingly, people don't value "bits" or information. We value moments and emotions and work and art. There's no successful current consumer business model for people to store and backup photos (Backblaze is mainly prosumer). And so aren't social networks the real backups by now? The redundancy of publishing on multiple services means some photos will stick and the rest will fade, somehow like former printed pictures I guess. Publish it or lose it?

walterbell 12y ago
> Local storage will not. External backup drives continue to increase in size at a falling cost per GB. It is cheaper and lower latency to store/retrieve a large amount of data locally than remotely, unless you believe that the current glut of "near-free cloud storage" will continue indefinitely. Market cycles suggest otherwise. > Publish it or lose it Why do social networks store your photos "for free"?
icebraining 12y ago
What happens when I don't have a computer anymore? Wifi-enabled drives are already common; no computer needed. They are even targeted at mobile device users ("for iPad/iPhone").
artmageddon 12y ago
> And so aren't social networks the real backups by now? I hope not. I cringe when I see people who treat them like they are. What do you do when your account password gets cracked(it happens to people on Facebook a lot), or the site itself goes away(MySpace)? Also, you're not able to store the images in their original form; I'm pretty sure sites like these will have limitations on what you can upload. I'm not a pro-photographer by any means, but there's no way I could upload a RAW file to Facebook.

rcthompson 12y ago

I would never consider using Dropbox (or any other cloud storage) as my sole backup system for critical files unless I was willing to pay for the "Packrat", i.e. infinite history, feature.

rayiner 12y ago

The bug in question: > From all this information it seems that Dropbox client first deletes files locally before it informs the server about the new selective sync settings. Consequently, if the client crashes or is killed before the server is contacted, the files remain deleted without any trace. After the client restarts again, it only sees there are some files missing and syncs this new state with the server. It concerns me that the top several comments are some variation of "you shouldn't trust the cloud with your data." How did software services become so totally insulated from the expectations we have of every other service that holds on to our valuable property?

kalleboo 12y ago
> It concerns me that the top several comments are some variation of "you shouldn't trust the cloud with your data." No, it's "you shouldn't trust one single thing with your data". Hardware fails. OSes have bugs (OS X's "space in your drive name and we delete your user folder.") Services have unexpected gotchas. This is why you never store anything in only one place. A sync service by definition is not a separate place (unless he had found a bug in the Packrat feature)
- rayiner 12y ago
  My point is that, for some reason, we have been conditioned to believe that bugs and data loss are just something that happen as opposed to something that service providers, who we pay money to, are charged with avoiding. We hold software to a much lower standard of quality than we hold other products and services.
  - kalleboo 12y ago
    * we have been conditioned to believe that bugs and data loss are just something that happen They're something that happen to unregulated services that cost $8/month. If I paid $8/month for electricity, water or Internet access, yeah I'd expect yearly blackouts. To get an SLA you have to pay more.
    - 12y ago

johnnyladders 12y ago

If it's not saved in three places, it's not saved.

zperrault 12y ago

I noticed there hasn't been any mention of attempting to recover your deleted files from your local disk. If you haven't tried it yet it is worth a shot. There are a few software solutions out there that will scan your disk for files that have been deleted. On Windows I have used Piriform's Recuva[1] and on OS X I have used PhotoRec[2], both have worked rather well. It is worth noting that the longer you wait to try the more likely the sectors are to be overwritten with new data. (And you should back up any files currently on the disk). I know it is a long shot at this point but some chance is better than none. Good luck. [1] http://www.piriform.com/recuva http://www.piriform.com/recuva [2] http://www.cgsecurity.org/wiki/PhotoRec http://www.cgsecurity.org/wiki/PhotoRec

dpcan 12y ago
Or, I wouldn't be surprised if tomorrow he posts that he found them all in his trash.... Because nobody ever thinks to clear it. Sometimes the simplest solution is right under our nose.
- pyre 12y ago
  He said that he de-synced them to free up space. I'm assuming that he would have noticed if the operation didn't free up space.

JoeAltmaier 12y ago

My buddy Tom says "If its not on your current computer, its lost". Imagine when Dropbox closes one day (as every company ultimately does), how much will be lost.

therealmocker 12y ago
Since all the files are sync'ed to the computer, nothing would be lost unless selective sync was enabled.
- rockdoe 12y ago
  A comparable bug would be that when the server goes away (Dropbox closes), the client starts deleting all local files.

steven2012 12y ago

In today's digital age, I believe lots and lots of people will lose a lot of valuable information like photos because they don't have a good understanding of what backing up means. I've warned my friend who is computer illiterate that he is risking losing his childrens' complete photo collection for their entire lives because he is only keeping a single copy of his data on his hard drive. This is probably the situation with millions of people in the US alone, and when that hard drive failure event happens, which will happen, then they will lose everything.

DanBC 12y ago
Did that work? Would it have been more effective to just provide him with a list of stuff - a choice of two drives; a choice of two softwares (with really easy URLs to buy and download) and instructions for using and testing. That could be a gift for birthday or Xmas?
- steven2012 12y ago
  I bought him a new hard drive, copied the photos over and gave him instructions on how to do it. He hasn't bothered to update his backup once, and it's been about 5 years and his photo collection has only grown.
  - kngspook 12y ago
    I'm thinking of buying a CrashPlan Business account (my corp uses the Enterprise version) for my family for this reason. I set them up with TimeMachine, but even plugging in a USB drive has proven to be too much to remember.
ominous_prime 12y ago
This is becoming even more important with SSDs. I used to do a lot of data recovery, and photos were of course a very common type of data that customers wanted to recover. If the drive was spinning, I could usually recover most of the data. Even if the drive was completely dead, there was still some chance at recovery of I had the correct logic board, and for a hefty price I could send it out to a company with more resources to get the platters spinning again. SSDs can be much harder recover from, since you need information stored in the controller to map out the data, and many consumer level hard drives brick themselves on failure to prevent silent data corruption.

servowire 12y ago

The solution to this kind of problems is: Make Read-only "disaster recovery" backups - zip it and burn it to a DVD set or Bluray, and put it off-site. Make incrementals or differentials of those discs every few months and a full data set each 3 months. Takes a few hours, I like to burn all my data to Bluray (50GB per disk) while watching a movie. Store the bluray data sets at a friend's house and never worry again.

filmgirlcw 12y ago

I had an issue similar to this (but fortunately not nearly as bad) about two and a half or three years ago and that was what convinced me to pay of the Packrat service add-on. I agree with the OP that that should be a built-in option for all paying customers -- or at least make it more visible as an add-on. I've had instances in the past where it was months later that I realized something was either deleted or didn't sync and I had to look through Events and use Packrat to restore. It can be scary - even if you do make backups of your backups.

danbee 12y ago

The moment you only have one copy of your files anywhere you don't have a backup. I've heard it said that if a file doesn't exist in at least three different places it might as well not exist.

kngspook 12y ago
Exactly. (I can't believe I had to scroll down this far to find this.)

vondur 12y ago

I worry about stuff like that happening to me with my photos. The best idea I've come up with is setting up a Synology that I export my photos to. It has a basic web app for showing off photos. I then use it's AWS S3 service to back those photos up every night. The Synology also has something like RAID1 on it for some redundancy at the local level. The hard part is remembering to export the files from the computer to the Synology.

mhartl 12y ago

Thanks for the tip on the Packrat feature. I didn't know about that. Added.

je42 12y ago

This is clearly a problem for the reputation of dropbox. On their website they market the service as: "Even if your phone goes for a swim, your stuff is always safe in Dropbox and can be restored in a snap." With this message in mind, and a valuation of roughly >>10$ BILLION<<. I think, you can have very very high expectations regarding the data retention of their service. Also, the whole point of using a service like dropbox is to remove friction/time investment regarding the standard backup/sync tasks. If you suggests to handle multiple harddisk backups locally+offsite, that is fine except that not everyone wants this kind of time investment and cost associated with a do-it-yourself backup service and rather depend on a service provider that they can trust.

wrs 12y ago

As I said to someone the other day after dealing with yet another failure by Dropbox to do the safe thing or even the reasonably-recoverable thing with our data: "One reason I'm not worth a billion dollars is I would never have let Dropbox ship like this!" Doing sync on top of an existing filesystem, making it work with existing apps, maintaining solid transaction semantics, and having a simple, understandable user model -- you just can't do all four. The Dropbox experiment has now shown that "correctness" doesn't matter, commercially speaking.

PhantomGremlin 12y ago

Dropbox has one of the worst websites on the Internet. Seriously! E.g. people have mentioned that "packrat" would have helped in this situation. But try finding that service from the pricing page. Hint: NOT THERE. Someone here linked to some explanation in their "help" section. Seriously? Such an important feature is only mentioned under "help"? Even if you go to sign up for the paid version, Dropbox Pro, packrat isn't mentioned [1]. Their horrid website alone is enough for me to not want to do business with them. Someone else here has succinctly summarized the situation [2]: One reason I'm not worth a billion dollars is I would never have let Dropbox ship like this! [1] https://www.dropbox.com/upgrade https://www.dropbox.com/upgrade [2] https://news.ycombinator.com/item?id=8105548 https://news.ycombinator.com/item?id=8105548

Fofer 12y ago

Once you deleted your own local copy, the folder of photos on Dropbox was no longer a "backup." You just moved your one copy elsewhere. You essentially had NO backup. And everyone should know, you should always have at least two backups. (Again, you had zero.) I wouldn't even consider Dropbox (without Packrat) a valid backup destination either, because any changes get synced. Sorry this happened to you, but better backup practices could have avoided it easily.

ecnahc515 12y ago

I use both backblaze, as well as Google Drive. I don't have anything in particular I'd be completely upset if I lost it forever, which is why I dont do local backups, but I've definitely been looking at using Crashplan as a secondary backup. I just cant be bothered to hook my laptop up to an external when I hardly sit next to it anyway. What does HN think? Safe if I went with 2 cloud backup providers?