9 ms·
Hard Drive Stats for Q2 2016
- JoshGlazebrook 10y agoLooks like Hitachi (HGST) is still leading in terms of reliability. I actually have a 400GB Hitachi drive that came in my Dell Dimension 9100 a little over eleven years ago. It has a few bad sectors but it's still chugging along just fine. Looks like it's this one: http://www.newegg.com/Product/Product.aspx?Item=N82E16822145067 http://www.newegg.com/Product/Product.aspx?Item=N82E16822145...
- slededit 10y agoI do wish they'd spend more time on their client. I choose them specifically because of how they publish results like these, but when your install can get corrupted like this [1], it doesn't inspire confidence. 1. http://imgur.com/a/3XmEL http://imgur.com/a/3XmEL
- rickyc091 10y agoYeah, their software was really buggy so I ended up changing to crashplan. I've been using them for years without any regrets.
- dagw 10y agoHeh. I switched my wife from crashplan to backblaze about 18 month ago because I was tired of the crashplan client being buggy. Backblaze has just worked.
- hrrsn 10y agoI don't like CrashPlan very much, but my backups are way faster to them than they ever were to Backblaze. My home connection has 200mbit upload, so I don't think the bottleneck is on my side.
- kalleboo 10y agoFunny, Backblaze backs up faster than my Apple Time Capsule. I'm on a 1 GBps home connection. I guess it all depends on your peering.
- dagw 10y ago100 Mbps here, and backblaze uploads are a good 50-100% faster than crashplan
- brianwski 10y agoBrian from Backblaze here. If you want to use more bandwidth in Backblaze, after you install find the Backblaze Control Panel, open the "Settings..." dialog, go to the "Performance" tab and UNSELECT "Automatic Throttle". Slide the manual slider all the way to the far far right. Then dial up the number of threads to 4 or 6. Seriously, you don't need to go any higher on the threads, after we use 100% of your bandwidth you can't get it to backup any faster. :-)
- hrrsn 10y agoI suspect that latency has something to do with it. Crashplan's servers are a good 100ms closer to me.
- thescriptkiddie 10y agoI had to stop using crashplan because there is no way to roll back revisions within a file. If a file gets corrupted it'll just back up the corrupted version on top of what was previously a good backup.
- barrkel 10y ago? That's not what I see in crashplan. Expand the tree view until you get to the file level, and then expand the files: it shows the times of the backup revisions. You can also specify a target time to restore to at the bottom of the restore tab.
- jarrodatCode42 10y agoWanted to pop in here and expound on the topic. CrashPlan has versioning that is set (by default) to back up changed files in 15 minute increments. If a file is corrupted, CrashPlan will attempt to back up the changes as it is not a corruption detection utility. However, it's versioning allows for a user to easily restore the uncorrupted file by selecting a date prior to the corruption occurring. This process is explained in greater detail here: https://support.code42.com/CrashPlan/4/Troubleshooting/Recovering_Files_Infected_By_CryptoLocker_Or_CryptoWall https://support.code42.com/CrashPlan/4/Troubleshooting/Recov... The default versioning can be adjusted, so ultimately it's the responsibility of the individual user to confirm the health of their backup and verify the versioning frequency that best fits their needs.
- swinglock 10y agoThe CrashPlan client is horrific but at least it runs on Linux unlike Backblaze. I was hoping Backblaze was better, too bad.
- haasn 10y agoWait, wouldn't that be standard? I've never used online backup, but I would naively assume that Linux would just have a block device or FUSE driver for these things. How can that not be the case?
- anonbiocoward 10y agoI'm not sure, what would a block device have to do with diff'd filesystem backups? And what would FUSE add? I'm pretty sure the CrashPlan client a .jar, so you just need a jvm, thus broad OS compatibility.
- haasn 10y agoOn second thought, block devices are sort of sketchy for this use case - but FUSE still seems like a perfectly natural way to export networked storage. I'd expect a backup hierarchy to just be a directory of read-only snapshots of my files - I mean, this is how I organize backups locally either way (e.g. /snapshot/YYYY-MM-DD/etc/whatever.conf). The advantage of using a standard API like filesystems and FUSE is that you can use it easily and from the terminal without needing to deal with bloated GUIs and JVMs that are no doubt hard to install, hard to use and hard to rely on. A JVM is also a pretty hefty requirement to have on a machine. (I personally avoid installing it all costs, mostly because everything that targets JVM seems to be crud to begin with)
- click170 10y agoThis really kills me about Backblaze. I'd love to support them, I'd prefer no other backup vendor. That's why this is so hurtful: https://help.backblaze.com/hc/en-us/articles/217664628-Is-Backblaze-going-to-offer-Linux-support- https://help.backblaze.com/hc/en-us/articles/217664628-Is-Ba... Spoiler alert, the answer is a surprisingly firm no. Not even an optimistic "we're working on it!". Doesn't even have to be "Linux support", just open an API, we'll do the rest! Use quotas for API users if the concern is around abuse, but please help us Linux folks use your service! Please!
- pcurve 10y agoWhat are we exactly looking at here? That is somewhat frightening how busted it looks.
- slededit 10y agoThe results of my attempt to try them out about a month ago. These are screenshots I sent to their tech support.
- atYevP 10y agoYev from Backblaze here - Looking at those screens that's what happens when the installer doesn't install correctly (obviously), it can happen for a bunch of reasons, usually the easy stuff like anti-virus and all that jazz, but usually clears up if you reboot and retry.
- deleted 10y ago[deleted]
- toast0 10y agoThat doesn't look that busted to me, it appears that something prevented localized strings from loading, so we're just seeing the translation tags. It's not surprising that they don't fit in the UI, cause they're kind of long -- probably much larger than the localized strings are.
- brianwski 10y agoBrian from Backblaze here, you are absolutely correct. I'll copy and paste my answer from above to here: The failure in that picture is that somebody deleted the file found at C:\Program Files (x86)\Backblaze\bzbui_interface.xml That extremely simple file contains all the strings for the Backblaze GUI in all 11 languages. What your screenshot shows is what happens when that file is missing. If you run a Backblaze installer over the top of that installation it should repair that by placing the latest version in place as it always does. If you STILL cannot get that file installed, something is going entirely, totally hog-wild wrong with your system. Hundreds of people install every day and that file is always present. If you were running along and suddenly that file disappeared from your computer, I would stop and prepare a full restore from 15 days ago from Backblaze. This is totally free. When files randomly disappear from the hard drive, one cause might be your hard drive is starting to die.
- TeMPOraL 10y agoWell, one has to grant them that they at least did use meaningful labels for their strings. You can figure out how to use the app from that alone.
- brianwski 10y agoBrian from Backblaze here. The failure in that picture is that somebody deleted the file found at C:\Program Files (x86)\Backblaze\bzbui_interface.xml That extremely simple file contains all the strings for the Backblaze GUI in all 11 languages. What your screenshot shows is what happens when that file is missing. If you run a Backblaze installer over the top of that installation it should repair that by placing the latest version in place as it always does. If you STILL cannot get that file installed, something is going entirely, totally hog-wild wrong with your system. Hundreds of people install every day and that file is always present. If you were running along and suddenly that file disappeared from your computer, I would stop and prepare a full restore from 15 days ago from Backblaze. This is totally free. When files randomly disappear from the hard drive, one cause might be your hard drive is starting to die.
- slededit 10y agoSince your willing to look at this now, here is the install log: http://pastebin.com/MB6dEiZD http://pastebin.com/MB6dEiZD Looking at it, its clear it had some problems. The issue is the installer said it finished successfully.
- tkinom 10y agoI love to see similar stat for SSD. Maybe DigitalOcean can publish their SSD failure rates if any.
- retrack 10y agoThis study from Google is interesting already: http://0b4af6cdc2f0c5998459-c0245c5c937c5dedcca3f1764ecc9b2f.r43.cf2.rackcdn.com/23105-fast16-papers-schroeder.pdf http://0b4af6cdc2f0c5998459-c0245c5c937c5dedcca3f1764ecc9b2f...
- honkhonkpants 10y agoThe problem with the Google information is their rig is full-custom everything: silicon, firmware, and interface. It can't possibly influence your buying decisions.
- olavgg 10y agoPatrick from ServeTheHome recently posted that of the 160 used Intel SSD drives he had bought from Ebay. Not a single one has failed yet. Intel, for example, advertises a 0.2% AFR on its data center SSDs which is much lower than observed hard drive failure rates which is 3-5%.
- DKnoll 10y agoCan somebody tell Seagate what SSHD means.
- kirian 10y agoThis the the acronym Seagate uses for solid state hybrid drives - http://www.seagate.com/gb/en/solutions/solid-state-hybrid/ http://www.seagate.com/gb/en/solutions/solid-state-hybrid/ I don't think Backblaze uses any of these so likely they've just pulled a stock photo of a hard drive from somewhere.
- DKnoll 10y agoI know what they mean, but the initialism has another well known use so it's annoying. Not to mention that a device being solid state by definition precludes it from being a hard disk. Logically it just makes no sense to me. shakes fist get off my lawn!
- jimminy 10y agoTo clarify what dknoll is talking about: 'sshd' is the reference command to the 'ssh daemon'. To dknoll, SSHD for the Solid State Hybrid Disk, means that it has components of both drive systems, spinning platters and a functional amount of NAND for feequently used applications. You could use it purely for it's SSD assuming the drive controller continues to function. Though that is a ridiculous proposal.
- DKnoll 10y agoI understand what the device does and why they named it as they did. It's an awkward use of language, that's all I'm pointing out. I think 'Solid State Hybrid Disk' is a syntaxically incorrect name as it suggests the entire device is solid state when it is not. Hybrid State Disk perhaps?
- jimminy 10y agoI read it as "Solid State Hybrid" -Disk. Which when regarded under the conventional drives at the time it was announced being primarily HDD's it makes it more clear. You have an HDD that is a hybrid with solid state components. Logically clean cut, imo.
- advisedwang 10y agoIt looks to me like this analysis is ignore the age of drives. Specifically it might be comparing the current failure rate of years old drives against the failure rate of months old drives. This is likely to make newer disks look much better. I'd love to see some attempts at calculating mean time to failure (which is admittedly difficult to estimate until you have had a disk long enough to see a whole generation of them fail).
- jxcl 10y agoIn case anybody missed it, Backblaze publishes their raw data, so it should be possible to do this analysis: https://www.backblaze.com/b2/hard-drive-test-data.html https://www.backblaze.com/b2/hard-drive-test-data.html
- cm3 10y agoI used to shy away from Seagate drives, and even though the stats here are for 3.5", it doesn't look like WD drives are more reliable than the Seagate. I was planning to get a 4TB 2.5" portable usb drive, and right now it's just Seagate that offers one without scary features. WD has one but not the Elements, just the MyPassport which includes the risky AES feature, and I don't want to accidentally lock or encrypt the disk because of that and then need a Windows tool. Would anyone advise against getting a http://www.seagate.com/consumer/backup/expansion-portable/?sku=STEA4000400 http://www.seagate.com/consumer/backup/expansion-portable/?s... drive?
- Veratyr 10y agoSomething important that I don't think many people bring up is the cost of these drives in a RAID, taking reliability into account. When you're using this as the sole drive in a desktop machine or something like that, paying the extra for a HGST is pretty straightforward, but I see too many people passing over Seagate or WD when they intend to put them in a RAID, for "reliability"'s sake, without (it seems to me) much thought into whether it's really worth it. Sorting by $/GB for Hitachi, Seagate and WD on drives >= 4TB (http://pcpartpicker.com/products/internal-hard-drive/#m=19,34,38&sort=a7&page=1&S=4000000,10000000 http://pcpartpicker.com/products/internal-hard-drive/#m=19,3...), you get: - 4TB WD: $129 - 4TB Seagate: $114 - 4TB Hitachi (HGST): $158 So in order to get the reliability of a HGST, you're paying a 38% premium, is the extra reliability helpful in a RAID? If you look at say a RAID6 array, aiming for usable storage of say 16TB, HGST's longest running drive (at 0.11% failure rate) would have a roughly 0.000016% chance of failing in a given year (0.0011^3 * 6 * 5 * 4) (chance of first drive failing = 6x(single), second=5x(single) etc.), assuming nobody replaced a failed disk, since you need 3 drives to fail. Now an array of Seagate's longest running drive (2.66% failure rate) of the same size would have a chance of failure of ~0.225853% in a given year. But, the cost of the disks in the Hitachi array would be 6x158 = $948 while the price of the disks in the Seagate array would be 6x114 = $684. For the price of the Hitachi disks you can get 8 Seagate disks. So what happens if you just add the extra Seagate disks to the array as extra parity? Now you need 5 drive failures, giving an equation that looks like ((8 * 7 * 6 * 5 * 4 * (0.0266^5)) * 100), and a chance of failure of 0.008949%, still way higher than the Hitachi. In the end buying Hitachi/HGST seems like the right choice anyway but I thought it interesting since I hadn't seen anyone else look at things this way. If anyone has any problems with my math, please feel free to point it out, my stats background is pretty limited.
- haasn 10y agoI think a better way to look at it is not the chance of total data loss, but the expected running cost over time. Even with just RAID6 it's hard to imagine a scenario in which you would be unable to replace one of the failed drives and resilver before getting two further failures, so the naive reliability calculations are extremely biased. So if we assume that we can safely operate at some fixed level of redundancy (say 3 copies) across all hard drive vendors, the only question is (how many drives you need to replace per year) × (the price of those drives).
- asher_ 10y agoI love these writeups from BackBlaze. Does anyone know why some of the confidence intervals around the ARF are asymmetric?
- donatj 10y agoI love and look forward to these. I bought the drives for my NAS based on the previous one.