13 ms·
Backblaze Hard Drive Stats for 2016
- overcast 10y agoAlways amazed on how far ahead HGST is in reliability. What are they doing that is 2-6x better in failure rates?
- theandrewbailey 10y agoHGST was originally IBM's hard drive business (after selling it to Hitachi, and now WDC). There's probably some heavy duty/reliability traits that's still left in the manufacturing and company culture.
- overcast 10y agoThe only thing I remember from IBM is the DeathStar(Deskstar) drives, that ALL failed on me. Even the garbage refurbs they sent back.
- rb808 10y agome too. I wonder if there is a cycle where companies have a terrible reputation then overcompensate, presumably they'll slowly get worse reliability from here on as they focus on costs instead of failure rates.
- maxerickson 10y agoIt would also be interesting to see if the personnel involved in the engineering correlated with failure rates.
- deleted 10y ago[deleted]
- jandrese 10y agoYep, it was especially bad because IBM had the gold standard for drives before that point. They were more expensive, but you paid for the reliability, so people got really angry when they started failing regularly. From what I can tell, they fixed those problems and went back to the high quality but slightly more expensive drives of old. The most interesting part is that Backblaze still buys mostly Seagates, simply because they are easier to buy in bulk and have a better price. Even though they fail more often the failure rate isn't bad enough to be a problem. Also, Seagate's failure rate has been decreasing as they release new drives.
- tw04 10y agoHitachi, in general, has always been known for bullet-proof storage. I was extremely sad to see them sell off that business. Hopefully spinning drives are dead before WDC culture can bring HGST back to mediocre reliability.
- kogepathic 10y agoIt's especially interesting because Western Digital bought HGST in 2012. [0] To my knowledge, there have been innovations in the spinning rust (ahem hard drive) market since 2012. So why is HGST, a WD subsidiary, still making much more reliable drives? Different design teams? HGST factories instead of WD? [0] http://m.theregister.co.uk/2012/03/09/wd_closes_hgst_buy http://m.theregister.co.uk/2012/03/09/wd_closes_hgst_buy
- deelowe 10y agoHGST was acquired by WD, but they have very different manufacturing processes and AFAIK the design teams are still seperate.
- handruin 10y agoI've gone all-in on HGST for personal use because I've seen their results from Backblaze and also have good luck with them so far. I have 14 of their 4TB NAS and 22 of the 6TB NAS.
- Symbiote 10y agoI'm amazed that personal use requires 188TB of storage. That's more than my employer's Hadoop cluster...
- overcast 10y agoSmall hadron collider?
- userbinator 10y agoPerhaps a small hardon collider. (Context: http://archive.fortune.com/2006/11/30/magazines/fortune/obrienseagate.fortune/index.htm http://archive.fortune.com/2006/11/30/magazines/fortune/obri... )
- nerdponx 10y agoI was debating whether I should "favorite" this thread. You made my decision for me, thank you.
- avn2109 10y agoMaybe he does his own offsite backups with "personal use" HD's in several physical locations, each in a highly-redundant raid configuration? Even so it seems like a lot.
- antisthenes 10y agoSomeone's gotta back up that climate data...
- coralreef 10y agoWas not surprised to see Seagate leading the failure rate. Had a portable 2TB seagate drive fail within weeks. There goes my vacation photos and videos!
- handruin 10y agoI had four of the Seagate 3TB drives that are listed in their ratings with high failure rates. All four of mine died in less than 3-years. They were a really disappointing purchase.
- Justin_K 10y agoSame - had two and both died. No recourse for an obviously defective product.
- aaronmdjones 10y agoI had 4 Seagate drives of varying capacity and model numbers die on me within 11 weeks of each other, all were less than 3 years old and one of them was in use for less than an hour. Never again.
- theandrewbailey 10y agoIt looks like more recent models have a lower failure rate than WD's.
- disiplus 10y agomy two seagate drives in my nas failed, one in 09/16 and one 01/17. thank god for raid. i replaced them with bigger seagate drives. maybee i should not.
- digler999 10y agoremember raid != backup. you and everyone who has valuable data should have offsite storage separate from your nas.
- 10y ago
- PascLeRasc 10y agoDoes anyone have any insight into why they don't have any 10TB drives running?
- atYevP 10y agoYev from Backblaze here w/ some insight -> Yes! They are expensive. We are looking to test some in our environment very soon, but may not have enough to (we want at least 45) to truly test them. Once they drop further in price and we can afford to drop them in some of our pods we'll be all over it!
- PascLeRasc 10y agoThanks Yev! I'm looking forward to those. I didn't realize they were that much more/TB than the 8TB guys.
- atYevP 10y agoYea still pretty pricey! Getting there though! We're waiting :D
- newman314 10y agoIs there a graph somewhere that shows the price/GB vs drive capacity that you would suggest? I've briefly tried looking for this in the past before but was never able to find a well maintained source.
- atYevP 10y agoIt's tough b/c we don't really keep track of prices after we buy them. Sometimes we go out and look up different drives on Amazon and add those in, but I don't remember if we keep track of exactly what you're trying to find - sorry! You could always look at the drive model and do a quick search and cross-check with our rates. Just know that your environment is different from ours so it might not be 1:1 :)
- 10y ago
- softawre 10y agoFunny how it is this exact blog post, every year, that really sets it in for me how fast the last year has gone by. Thanks for that, I guess.
- runjake 10y agoAre there any reasonably scientific "death matches" like these for 2.5" spinning hard drives? Edit: clarification
- DiabloD3 10y agoThe 2.5" market has gone almost entirely SSD. I, too, would like to see more regular SSD death matches.
- cm2187 10y agoParticularly given that almost no SSD review says a single word about reliability, which is my principal concern.
- wmf 10y agoReviewers can't really believe anything the vendors say and can't afford to run hundreds of SSDs for years to gather real data so there's not much they can say. By the time there is real-world experience with an SSD model it's probably obsolete anyway.
- cm2187 10y agoYep. But that's a problem...
- throwawayish 10y agoThere isn't really a compelling reason to run significant storage off 2.5" HDDs anymore -- before SSDs 10k or 15k 2.5" HDDs made sense (access time), but nowadays...
- runjake 10y agoHere's a compelling reason: cheap, portable mass storage. Eg. USB3/Thunderbolt-powered multi-terabyte drives that fit in a backpack and don't require external power.
- Twirrim 10y agoBackblaze's environment is abnormal. They do very little to dampen vibration amongst other things, based on their last pod design. That puts drives under stresses that don't reflect what you're likely to see in your environments. What you're seeing is not the most reliable, but the most reliable under their particular set of bad conditions. The reliability under your circumstances might be wildly different. Their pod approach even results in worse performance: more vibration results in more effort spent by the drives trying to keep the head positioned in the right places, increasing latencies and decreasing read performance. The higher density they push for with their pods actually compounds this. Drives vibrate. Fans vibrate (chassis, PSU, CPU, the works) . If you don't pad them all appropriately, the vibration of the components ends up causing the entire rack to vibrate. By the time you've got several dozen of drives in the rack, the whole rack will be feeling the effect.
- Symbiote 10y agoDoes an average server have much to dampen vibration? Something like [1]. I occasionally replace drives in similar machines, and I haven't noticed anything that looks like it helps with vibration -- but I don't look very much. [1] http://www.dell.com/us/business/p/poweredge-r530/pd?ref=PD_OC http://www.dell.com/us/business/p/poweredge-r530/pd?ref=PD_O...
- Twirrim 10y agoAverage servers don't tend to have much. Storage servers do tend to do stuff to dampen vibration, but then it's a question of storage density. When you've got something like a backblaze pod that's 60 drives in 4U of space. It'd be more unusual to have 60 drives in total in a rack full of normal servers. Now consider 60 drives / 4U, and you'll have multiple of these storage pods in a rack. It adds up.
- jasode 10y ago>What you're seeing is not the most reliable, but the most reliable under their particular set of bad conditions. The reliability under your circumstances might be wildly different. You do have a valid point but I also wanted to add that my harddrive failures in my gentle home environment matches BackBlaze's more hostile environment. I have 50+ harddrives in various capacities from 2TB to 10TB and my differences from BB include drives being iso-mounted on rubber grommets and they also do not run 24/7. (I only power them up when I need archived files off of them.) Even with those stark operational differences, my harddrive failures exactly match the generalization of Backblaze's data: More failures with WD Green and Seagates especially the 3TB drives. My HGST drives purchased over the last 4 years have zero failures so far.
- izzydata 10y agoWhy do they buy such weird proportions of each type of harddrive? Do they just use anything they can get their hands on? If the purpose was actually to test drive failure rates wouldn't you want to buy similar amounts of every drive?
- andrewbinstock 10y agoDrive failure rates can be computed across different batch sizes, provide the batches are big enough to provide meaningful numbers. There's no need to have the batches be exactly the same size to get useful and accurate data.
- wcfields 10y agoThey typically purchase the best $ per MB. Even "shucking" external hard drives bought at Costco after Thailand experienced a flood that took offline several hard drive manufacturing plants. [1] [1] https://www.backblaze.com/blog/backblaze_drive_farming/ https://www.backblaze.com/blog/backblaze_drive_farming/
- cm2187 10y agoDoes anyone has any experience on the reliability of the Seagate 8TB archive drives? They are incredibly cheap for cold storage, but a bit concerned by the Seagate numbers in the Backblaze stats...
- bkruse 10y agoYes, I run about 800 of them now. They have not been as reliable as the 6TB archive drives. I think this was related to a bad run that seagate had while manufacturing. Our numbers, overall (with 6,000+ drives now, if I remember correctly) are much lower than Backblaze's reported failures. We also classify a drive as failed when it throws ANY SMART error, then it's diagnosed with Seagate's internal tool (which I believe they are open sourcing, if they haven't already) and put back into production Seagate has given us good warranties on these drives as well. Yes a higher immediate failure rate (first 1 month) with the 8TB drives was annoying, but they more than made it right replacing the drives.
- cm2187 10y agoThanks!
- mnw21cam 10y agoMy anecdata is that I bought one for home use, and it failed after a couple of months. I sent it back for replacement, and the replacement failed after a couple of months. So I gave up, and got something else instead.
- gravypod 10y agoI'd love a set of charts made that would plot Hours Runtime vs Failure Rate and Data Transfer Rate vs Failure Rate. These tell us little about how good these drives hold up, it tells us more of the churn cost that backblaze has.
- E6300 10y agoI love these statistics, but I would just like some clarification on what exactly constitutes a "failure". Does the drive have to stop responding completely, or are a few bad sectors enough to consider the drive "bad"?
- user5994461 10y agoRelated topic: I'd like to see an update on the enterprise vs commodity hard drives. A few years ago, the consensus (and data) was that they were mostly the same thing and it was arguable to pay double for the enterprise one. But... in recent years, the consumer market have moved toward ever cheaper, slower and power savings hard drives (e.g. reduced RPM and stop the disk when unused for 30s). That calls for a re evaluation of the situation.
- throwawayish 10y agoMore importantly: shingled recording slows things down as well.
- bluedino 10y agoJust deleting/re-writing, yes? I remember reading they were good for storage situations where you just fill the drive with writes and pretty much just read from it for life
- atYevP 10y ago> Related topic: I'd like to see an update on the enterprise vs commodity hard drives. Yev from Backblaze here -> we wrote about this in 2013 and have honestly not bought too many enterprise drives since, simply because they were more expensive and the benefit was negligible in our usage. Lately though Seagate has had a nice run of enterprise drives and they're well-priced, so we might be giving those more of a shot in the coming months!
- pooloo1 10y agoHow is the benefit negligible? You are running consumer drives that are not designed to be run at a 100% load 100% of the time. The drives are not meant to be spun up constantly; in fact, many of them have mechanisms designed for them to spin down between use and lower power consumption. It's like complaining about a drag racer not being able to haul a ton of bricks without breaking the transmission on a regular basis.
- devwastaken 10y agoI remember a bunch of people saying that seagate's drives with 15% failure rates were a 'one time thing'.
- ents 10y agoHas anyone made a failure rate vs current price chart for all drives included in these reports to help guide purchasing decisions for normal people? I'd love to find where the dollar per terabyte sweet spot is, with respect to reliability.
- frik 10y agoWho owns the company that produces the HGST 3.5" drives? (the former Hitachi drives) Is it Western Digital? Is it Toshiba? someone else? (afaik the HGST 2.5 and 3.5 production lines were sold to different companies -right?)
- fnord123 10y agoWDC owns HGST.
- currysausage 10y agoIt's complicated: https://news.ycombinator.com/item?id=10057519 https://news.ycombinator.com/item?id=10057519
- frik 10y agoThanks a lot, it's indeed complicated.
- rbirnie 10y agoHow are you generating the raw data? Can you share your scripts that are used to generate it? I'd be very interested in generating raw data against the disks I maintain as well, might also be able to share it, which might be interesting since its primarily SSDs.