4 ms·

Given all the talk about how scalable some of these services are, they end up being pretty easy to break. I'm not on data hoarders, but I personally "broke" my

Given all the talk about how scalable some of these services are, they end up being pretty easy to break. I'm not on data hoarders, but I personally "broke" my crashplan account a year or two ago. All it took was having 3TB of data, mostly a bunch of family pictures, some home movies, a half dozen cloned git repos with a crapton of small .c files in them and having an account for >5 years. My client got wedged in some kind of server reindex operation (or whatever was happening on the server side in the datacenter) that took longer than the 24 hour window to complete. After they told me to reduce my retention window because I had to many "objects" it mysteriously fixed itself a week or two later after basically being offline for a couple months. Similarly some software I wrote a couple years ago, that wrote a bunch of small files into a single S3 bucket caused a lot of grief at a 3rd party S3 like service because they never imagined someone would write a couple TB of data into a single bucket (actually it used multiple buckets but each one was a TB or two of <1k files).

ValentineC 6y ago
> I personally "broke" my crashplan account a year or two ago. CrashPlan lost some of my data some time in 2012/13, blaming it on how they work with Windows's Volume Shadow Copy Service (VSS). I should have stopped my subscription there and then, but I continued subscribing for another 2 years, probably because I was anxious about needing something else that was in the file retention history. It wasn't a very good service anyway — slow to upload, often stopping mysteriously, and requiring 1GB of RAM for every 1TB backed up.
- StillBored 6y ago
  I effectively disabled their deduper early on. That solved a large part of the slow uploads, and as a bonus it also reduced the memory footprint. The latter of which is still a problem, but checking my machine at the moment. I see its consuming 10G of address space, but only about 600M are actually resident. I think the 10G number is something I tossed into the config at one point too. https://support.code42.com/CrashPlan/4/Configuring/Unsupported_changes_to_CrashPlan_de-duplication_settings https://support.code42.com/CrashPlan/4/Configuring/Unsupport... That means I've probably got a bit of duplication @ crashplan, but given the bulk of the data is a .jpg/mpeg dump from my cameras I don't care about the 5-10% of the archive which isn't compressed and likely has some duplication (the .git repos). OTOH, crashplan isn't my only backup plan, so if they explode tomorrow I don't think it will cause me any grief outside of having to find another offsite/cloud backup solution. (also as a further note, looking at my config log, it seems I need /proc/sys/fs/inotify/max_user_watches bumped pretty hard just to keep it running).

ValentineC 6y ago

> I personally "broke" my crashplan account a year or two ago. CrashPlan lost some of my data some time in 2012/13, blaming it on how they work with Windows's Volume Shadow Copy Service (VSS). I should have stopped my subscription there and then, but I continued subscribing for another 2 years, probably because I was anxious about needing something else that was in the file retention history. It wasn't a very good service anyway — slow to upload, often stopping mysteriously, and requiring 1GB of RAM for every 1TB backed up.

StillBored 6y ago
I effectively disabled their deduper early on. That solved a large part of the slow uploads, and as a bonus it also reduced the memory footprint. The latter of which is still a problem, but checking my machine at the moment. I see its consuming 10G of address space, but only about 600M are actually resident. I think the 10G number is something I tossed into the config at one point too. https://support.code42.com/CrashPlan/4/Configuring/Unsupported_changes_to_CrashPlan_de-duplication_settings https://support.code42.com/CrashPlan/4/Configuring/Unsupport... That means I've probably got a bit of duplication @ crashplan, but given the bulk of the data is a .jpg/mpeg dump from my cameras I don't care about the 5-10% of the archive which isn't compressed and likely has some duplication (the .git repos). OTOH, crashplan isn't my only backup plan, so if they explode tomorrow I don't think it will cause me any grief outside of having to find another offsite/cloud backup solution. (also as a further note, looking at my config log, it seems I need /proc/sys/fs/inotify/max_user_watches bumped pretty hard just to keep it running).