5 ms·
Hard Drive Reliability Stats for Q1 2015
- stephengillie 12y agoAre these consumer drives being used in an industrial setting? I'm surprised to see such high AFR from Seagate and WD. These brands have had decades of well-deserved loyalty and trust, and I'd never suspected some of their drives have such high failure rates. Why are HGST's AFR numbers so much more preferable?
- greggyb 12y ago>Are these consumer drives being used in an industrial setting? Yes, Backblaze is built around consumer hardware and a custom RAID(ish) setup.
- aidenn0 12y agoHitachi has consistently had the best AFR numbers from backblaze for a long time. As far as why other drives have such high AFR, historically Seagate and WD have had much better QA on their server class drives.
- stephengillie 12y agoI wonder if that's due to over-built drives (a quality standard independent of other factors) or if it's related to the specific controllers and other hardware in the Backblaze environment (independent of each manufacturer's drive quality).
- theandrewbailey 12y agoHGST was originally IBM's hard drive division. They probably have inherited much of the reliability secrets that the other manufacturers never bothered with for their consumer class drives (which Backblaze uses). Either that, or they learned a lot of stuff with the "DeathStar" model[0]. https://en.wikipedia.org/wiki/Hitachi_Deskstar#IBM_Deskstar_75GXP_failures https://en.wikipedia.org/wiki/Hitachi_Deskstar#IBM_Deskstar_...
- frik 12y agoYes, but nowadays HGST is no more. The company has been bought by WD, the 2.5" and smaller HDDs are now part of Western Digital. The 3.5" HDDs are now Toshiba due regulatory requirements. http://en.wikipedia.org/wiki/HGST http://en.wikipedia.org/wiki/HGST
- theandrewbailey 12y agoA surprising amount of industry consolidation happened after the Thailand floods a few years ago. There's what, 3 hard drive manufacturers left!? https://en.wikipedia.org/wiki/File:Diagram_of_Hard_Disk_Drive_Manufacturer_Consolidation.svg https://en.wikipedia.org/wiki/File:Diagram_of_Hard_Disk_Driv...
- Shorel 12y agoDamn. I'm still mourning the disappearance of Quantum. Those things were indestructible. But now my cellphone has more storage than these golden drives.
- Someone1234 12y agoThey have a blog post about the whole consumer Vs. enterprise drive thing here: https://www.backblaze.com/blog/enterprise-drive-reliability/ https://www.backblaze.com/blog/enterprise-drive-reliability/ The TL;DR is that the consumer drives had a better ROI since they failed at similar rates and were cheaper.
- madaxe_again 12y agoThe bit about the ST3000DM001 is fascinating - they've a longer article on it at https://www.backblaze.com/blog/3tb-hard-drive-failure https://www.backblaze.com/blog/3tb-hard-drive-failure - I have had four (out of the four I bought - two internal, two "shucked") of these die on me over the years, on the first occasion simultaneously in a raid-5 array that I'd just downloaded several TB of irreplaceable video to from SD cards. Now at least I know why.
- sliverstorm 12y agoSD cards are pretty easy to recover media from, if you haven't overwritten them. Also, I'm a bit surprised by their topology. Reading the bit about the 3TB drives, it sounds like three drive failures in their raid5 array would result in lost customer data. I hope they have a 2nd line of defense, e.g. tapes. One company I used to work for ran into sufficient failures (particularly of the living-dead variety, where a second drive fails during rebuild or after reboot) they made tapes the 3rd line of defense, and maintained two copies of all data at the cluster level, so that the total failure of an entire machine was acceptable.
- thaumaturgy 12y ago"A Backblaze Vault is comprised of 20 Storage Pods, with the data evenly spread across all 20 pods. ... Drives in the same drive position in each of the 20 Storage Pods are grouped together into a storage unit we call a “tome”. ... Every file uploaded to a Vault is broken into pieces before being stored. Each of those pieces is called a “shard”. ... Each file is stored as 20 shards: 17 data shards and 3 parity shards. Because those shards are distributed across 20 storage pods in 20 cabinets, the Vault is resilient to the failure of a storage pod, or even a power loss to an entire cabinet." From https://www.backblaze.com/blog/vault-cloud-storage-architecture/ https://www.backblaze.com/blog/vault-cloud-storage-architect..., which is well worth taking a few minutes to read. Their approach to data resiliency is pretty neat and full of the kind of common-sense reasoning that warms the very bottoms of my ol' sysadmin's heart ("use simple parts" and "build the tools you need to run your infrastructure", among others).
- userbinator 12y agoI thought it was the same article I'd seen before since I've seen this URL appear on HN before at https://news.ycombinator.com/item?id=8923535 https://news.ycombinator.com/item?id=8923535, but apparently it's not. Only the small "May 21st, 2015" text in the article is any indication. I think they really need to put a date in the URL and not just overwrite the previous version, so we can see how things change over time. Edit: in fact they had a nice bargraph of the failure rates by drive model in their previous post, but it's gone now. Fortunately someone seems to have archived a copy of it here: http://thetrendythings.com/read/28144 http://thetrendythings.com/read/28144
- theandrewbailey 12y agoSome of the previous articles are still up: https://www.backblaze.com/blog/best-hard-drive-q4-2014/ https://www.backblaze.com/blog/best-hard-drive-q4-2014/ https://www.backblaze.com/blog/hard-drive-reliability-update-september-2014/ https://www.backblaze.com/blog/hard-drive-reliability-update... https://www.backblaze.com/blog/what-hard-drive-should-i-buy/ https://www.backblaze.com/blog/what-hard-drive-should-i-buy/ https://www.backblaze.com/blog/how-long-do-disk-drives-last/ https://www.backblaze.com/blog/how-long-do-disk-drives-last/ You can also sift through the data yourself: https://www.backblaze.com/hard-drive-test-data.html https://www.backblaze.com/hard-drive-test-data.html
- Rexxar 12y agoWhat is the formula used for confidence interval ? "0.0%-106.9%" interval for low numbers is somewhat disturbing. What approximation cause this problem ?
- madez 12y agoYou can have an annual failure rate of over 100%. It'd mean that the expectation of the duration until all your drives failed is less than one year.
- wtallis 12y agoThe 106.9% isn't a problem of mathematics; it just means an average of more than one failure per drive per year, or alternatively an average lifespan of less than one year. It comes from the fact that they've only had those models in service for a very short time, and so they can only constrain the lifetime through the same reasoning used to estimate a MTBF of multiple years with less than a year of testing. With zero failures so far, it's obvious why the bottom of the confidence interval is at 0%.
- dragontamer 12y agoThey only have 50 Toshiba drives, which have only been used for a couple of months. With 0% failures over the 50 drives, you've got a statistical nightmare. You really have _no_ idea how reliable or unreliable the drives are. So the "confidence interval" is huge. The other drives have thousands and thousands of samples, and have been used for many years. They are confident in the measurement of the reliability. Toshiba has taken over the former Hitachi factories (along with HGST). So I'd expect Toshiba drives to be highly reliable. I hope that Backblaze can purchase more Toshiba drives so that they can gather some better statistics. For now, I'm confident in their 50 hard drives that haven't failed. That's a pretty good measurement, even if its not statistically reliable quite yet.
- frik 12y agoPlease rename the HGST drives. There are only Seagate, Western Digital and Toshiba - HGST is no more. Hitachi Global Storage Technologies (HGST) is now a wholly owned subsidiary of Western Digital! To address the requirements of regulatory agencies, in May 2012 WD divested to Toshiba assets that enabled Toshiba to manufacture and sell 3.5-inch hard drives for the desktop and consumer electronics markets. So all HGST 3.5" HDDs are now Toshiba, whereas the HGST 2.5" and smaller HDDs from Western Digital. See: http://en.wikipedia.org/wiki/HGST http://en.wikipedia.org/wiki/HGST
- atYevP 12y agoYev from Backblaze here -> They are run as separate entities: "It was agreed that WD would operate with WD Technologies and HGST as wholly owned subsidiaries and they would compete in the marketplace with separate brands and product lines.[4][5][6]" So, they compete with each other and use different supply-chains, so they are quite different, even if the money eventually flows in to the same conglomerate.
- frik 12y agoFrom where is your quote? If you read my explanation, and the wiki article you would see it is wrong. "Some of the HGST drives listed were manufactured under their previous brand, Hitachi. We’ve been asked to use the HGST name and we have honored that request." As I mentioned even if backblaze honors a request of Western Digital, it doesn't make the claim true. I researched on this topic some months ago, after I read backblaze previous article from late 2014 and bought several Toshiba (former HGST) drives. http://www.anandtech.com/show/5635/western-digital-to-sell-hitachis-35-hard-drive-business-to-toshiba-complete-hitachi-buyout http://www.anandtech.com/show/5635/western-digital-to-sell-h... mentions it too: "Western Digital will be allowed to acquire Hitachi’s 2.5” and SSD businesses, but not the 3.5” business. Instead Western Digital will be selling that business to Toshiba – factories and all – along with granting licenses for the necessary patents, which would allow Toshiba to effectively continue in the 3.5” market from where Hitachi left off. This would firmly establish Seagate, Western Digital, and Toshiba as the 3 major players in the hard drive business across all product segments." Confirmation is the press release by WD: http://www.wdc.com/en/company/pressroom/releases/?release=f8697da0-111a-49e8-a484-5266d18526ab http://www.wdc.com/en/company/pressroom/releases/?release=f8... So former Hitachi and HGST 3.5" drives are now Toshiba drives. "The 3.5" desktop and server Hitachi drives are all manufactured by Toshiba. WD wanted to buy Hitachi's drive division so there would be a WD/Seagate drive duopoly, European regulators said they had to sell Hitachi's 3.5" to someone else before they would approve the merger. Toshiba stepped up and got it. So yes it is somewhat confusing. You buy one of these drives and it will say "HGST a Western Digital company" on the box, when in reality it's all made by Toshiba, probably in an old Hitachi factory." -- a comment, from backblaze HN post in Sept 2014: https://news.ycombinator.com/item?id=8355860 https://news.ycombinator.com/item?id=8355860