20 ms·

What Hard Drive Should I Buy?

688 points by nuriaion 13y ago

aroch 13y ago
Heh...I, too, love 3TB WD REDs. At last count I have 40 of these racked up in servers or NASes and they've worked quite well for me
mustafab 13y ago
Too bad you still don't have linux client. Do you think supporting linux users anytime soon?
- Nickoladze 13y ago
  Relevance?
  - blisterpeanuts 13y ago
    Well, it's off the subject, but it's still an interesting question and I was wondering the same thing.
dnissley 13y ago
Interesting. I had always avoided Hitachi Deskstars after having heard they were nicknamed "Deathstars" for a reason. Perhaps that was once true, but clearly it's not anymore.
- michaelt 13y ago
  Apparently that was 2001 and they were IBM Deskstars at the time [1]. I guess this is why companies sometimes decide to rebrand/rename products. [1] https://en.wikipedia.org/wiki/HGST_Deskstar#IBM_Deskstar_GXP-model_failures_in_2001 https://en.wikipedia.org/wiki/HGST_Deskstar#IBM_Deskstar_GXP...
- yen223 13y ago
  I suspect those infamous disk failures were caused by faulty manufacturing runs, rather than some inherent flaw in the design.
  - radiowave 13y ago
    IIRC, it was specific to the 38GB model, and was traced to a batch of bad electronic components.
    - nknighthb 13y ago
      No, it was the whole 75GXP set (actually, according to Wikipedia, the 120GXP and 180GXP were also affected to a far lesser extent). The backlash was really nasty, though, because IBM's drives were something a lot of us had come to trust and rely on, the 75GXP was kind of a flagship drive, and IBM's handling of the issue was less than stellar. We felt betrayed. I never have bought another IBM (or Hitachi) drive. The reaction is visceral and intractable.
      - radiowave 13y ago
        Ah yes. I think I had the jumper on mine set to limit the drive to 38GB. Luckily, mine was OK.
  - justin66 13y ago
    I bought an IBM Deskstar and went through five returns over the course of a couple of years - across two different generations of drives! (The 75GXP and the 60GXP, I believe) I will say their warranty service was generally not bad... they seemed to get a lot of practice. The interesting thing is that what you say strikes me as almost certainly true, in a narrow way: there were probably some manufacturing runs that were a lot better than others. The proportion of bad runs to good runs is significant.
  - BashiBazouk 13y ago
    I worked in platter sputtering at IBM right out of College during the Death Star period. Basically, IBM had multiple parts manufacturing and sputtering facilities all over the world. The best disks/heads/motors always went in to the SCSI drives and the lesser parts went in to IDE drives. SCSI drives of that period were fairly bullet proof. Desktop IDE drives, not so much. Most of the San Jose and Munich production had higher yields and were mainly for SCSI and laptop drives. So it was less faulty manufacturing runs than the parts came from lesser production facilities, usually Singapore and the Philippines. They ended up closing the sputtering plant in the Philippines as they were never able to get their test yields up high enough...
- wmf 13y ago
  If you avoid Hitachi due to that bad batch and Seagate due to their bad batch and WD just because, you're left with nothing.
  - hga 13y ago
    To my understanding (see my other comment in this subthread for a bit more), neither Seagate nor WD in at least recent times (21st Century?) screwed up even close to that badly.
- hga 13y ago
  Well, there was a period where a company that failed that badly apparently lost so much business they were forced to sell it or the division. IBM Deathstars (which I missed buying my the most minuscule fractions time, phew) resulting a sale to Hitachi, Maxtor I gather to Seagate.
staticshock 13y ago
Interesting to see that kind of a difference between hitachi and western digital, given that WD owns HGST. Are hitachi drives marketed as higher reliability drives, or was the acquisition by WD simply too recent for the quality of the two brands to "equalize"?
- davis_m 13y ago
  Backblaze's data shows that the number of errors is largely related to the age of the drive. Because the older drives are from before Hitachi was acquired by WD, it is going to take a few more years for the brands to equalize if they do combine the manufacturing of both lines.
nwmcsween 13y ago
The worst service and hard drives I have ever had were from seagate. I had 3 separate hard drives RMA'ed _twice_ and all three failed within 3 months twice over, to this day I will not buy seagate.
DanBC 13y ago
Do the Thailand floods make any difference to this report? How reliable were those drives, and are the factories ba k up to full speed yet?
- yen223 13y ago
  The factories have been at full speed for the past year now.
  - pstack 13y ago
    You wouldn't know it by the prices.
    - izzydata 13y ago
      The prices are pretty much where they were from before the flood. Harddrive capacity has just gone up a ton. I bought a 1 terabyte before the flood for $60 and that is pretty much where it is at now if not cheaper in some cases.
      - gwern 13y ago
        Yes, but the problem is, hard drive $/GB has not recovered to the pre-flood trendline, even if prices have (4 years later) finally hit parity. The trendline has been badly hurt by the floods, it's pretty amazing. (I did a little tracking and was impressed how my predictions of the consequences were still insufficiently pessimistic: http://www.gwern.net/Slowing%20Moore%27s%20Law#kryders-law http://www.gwern.net/Slowing%20Moore%27s%20Law#kryders-law )
        muyuu 13y ago
        Economies of scale haven't recovered to pre-flood levels, because of the crisis in developed countries and because SSD is quickly growing its share of the market. These are completely different technologies and their R&D spending and amortisation is separate.
freshyill 13y ago
I wonder if any of this actually applies to consumer-grade drives. My wife's hard drive actually just died. It was 160 GB WD in a black 2006 MacBook. The drive itself was a replacement from 2007 since the original drive died just over a year into its life. Stupidly, since her Time Machine backup was misbehaving, I reformatted it and set it to start over. I spent the weekend recovering her data—with a lot of success, so no big deal. At any rate, this machine is long past its expiration date. It's time for a MacBook Air with an SSD, once the tax refund comes in.
- atYevP 13y ago
  Yev from Backblaze here -> All of our drives are consumer-grade. We try to avoid buying enterprise drives at all costs. These are all off-the-shelf internals and in some cases...externals that were made internal! :)
  - sehrope 13y ago
    > and in some cases...externals that were made internal! :) I remember reading about this a while back[1] (fun read!). Have you guys noticed a big difference in life expectancy of the repurposed external drives or do they generally match up with their internal equivalents? [1]: http://blog.backblaze.com/2012/10/09/backblaze_drive_farming/ http://blog.backblaze.com/2012/10/09/backblaze_drive_farming...
    - atYevP 13y ago
      Interestingly, we think they do as well if not better (http://blog.backblaze.com/2013/12/04/enterprise-drive-reliability/ http://blog.backblaze.com/2013/12/04/enterprise-drive-reliab...). We think it's because external drives are meant to be jostled around by Joe Schmo, so they do well in our storage pods, whereas internal drives and especially enterprise drives are meant to live in a perfectly Utopian environment.
  - lsc 13y ago
    Have you seen a different failure rate between the drives that were sold as internal vs. the drives that were sold as external and "shucked" ?
kylec 13y ago
This makes me feel better about the 4 HGST 4TB drives I just bought (http://www.newegg.com/Product/Product.aspx?Item=N82E16822145912 http://www.newegg.com/Product/Product.aspx?Item=N82E16822145...). They were the cheapest 4TB 7200RPM drives on Newegg by a non-trivial margin.
- atYevP 13y ago
  <3 <3 <3 <3 <3 Hitachi drives. If we (Backblaze) could get them at a rate within a few dollars of the other manufacturers at the 4TB level, we would solely buy them.
  - reitzensteinm 13y ago
    Surely not? If a specific model happens to have a large defect rate, having all your eggs in one basket could prove absolutely disastrous. Spreading your drives out, even among inferior options, seems like the only solid strategy.
    - atYevP 13y ago
      Normally diversification is absolutely the way to go, but once we've found a drive with low failure rates across the board, we'd want to move mostly to using that one drive type. We can always move over to another type of drive, but there's a very real cost of having to swap drives out as they fail, so the more reliable the drive, the more incentive to stick with it until it's no longer reliable. At the moment, we're more concerned about price, so we buy a wide-variety, but we have our favorites :)
      - scholia 13y ago
        If you're buying something to run for 5 years x 365 days x 24 hours, then how big a percentage of bucks is worth saving?
      - justin66 13y ago
        > Normally diversification is absolutely the way to go, but once we've found a drive with low failure rates across the board, we'd want to move mostly to using that one drive type. With regard to deployment, but also with regard to the drive reliability numbers you guys are putting out there: do you worry that variation between manufacturing runs is going to hose you? Do you find looking at the drives that you're getting a good variety of hardware from multiple manufacturing runs?
brokentone 13y ago
Very cool that Backblaze continues to post things like this. Few people have this experience. Of those who do, few (I assume) break it out to this level of detail, actionable for others. Of those who have good experience and records, most would consider it proprietary or just decide not to post. Kudos to Backblaze.
- atYevP 13y ago
  We're all on this crazy spinning ball together. Early on Backblaze made the decision to stay out of the hardware business (it sounded lucrative after we developed the v1 storage pod, but everyone was a software person, so we went that route), we like to pay our hardware findings forward. It's a good thing to do!
  - nickff 13y ago
    I really appreciate this; as someone who has had a number of SSD and HDD failures due to lackluster manufacturers (OCZ and Seagate respectively).
  - Zancarius 13y ago
    I really like this detailed write up. A certain large company in Mountain View suggested years ago that they had collected a great deal of metrics on consumer grade hard disks performing in a data center capacity, but refused to publish the data. The fact that Backblaze isn't embarrassed to share this with the rest of this gives me the warm fuzzies. Thanks for the hard work! I recently purchased a new Seagate, and I'm feeling a bit of buyer's remorse. That's not to say this article triggered the feeling completely (though it did push me over the edge). Taking this information and the knowledge that recent model Barracudas appear to have aggressive APM that loads/unloads the heads after a minute or so of inactivity (unless disabled, but it doesn't persist between reboots) AND seems to borrow from laptop-grade design... I think I'd be better off replacing it. Again. ;) I wonder if the poor longevity of Seagate drives is due to changes in quality control or design? I have older ones that don't utilize the APM load/unload features and are still running fine after many years. Then again, they were also pre-flood drives. I'd also be curious to know if there's a correlation with where the drive was manufactured and its survival rate, though I suspect that might be a much more difficult metric to collect.
izzydata 13y ago
Should the larger capacity seagate drives have a higher failure rate than small ones? That chart seems counter intuitive.
- unreal37 13y ago
  They said in the article that the 1.5TB drives were largely warranty replacements, and they suspect they were refurbs.
wnevets 13y ago
the WD reds only have 3 stars on newegg.com, interesting.
- zanny 13y ago
  Thing is most reviews are written immediately after recieving, and I'd rather get a defective drive in the mail and just RMA it on day 1 with the confidence when I get a working drive it will last longer than to have a drive that always works flawlessly out of the box (some 3tb seagate model I know got like 4.5 stars) but usually fails in a year. Though my current storage drive is a 3tb seagate one, that has been working fine for 2 years with no smart errors. Problem is I need to get the money to build a more appropriate backup raid for all my music / movies / games / etc, but that would be a pretty penny. I've just been using old dvd-rs (I mean, who uses those anymore?) as an impasse with anything really important on my 1TB external disks.
- whichdan 13y ago
  They have great reviews on Amazon, though.
- insertnickname 13y ago
  Also, the table in the blog post says their average life is 0.5 years... Maybe I misunderstood something.
olov 13y ago
"If the price were right, we would be buying nothing but Hitachi drives." I don't understand why they don't. Are the Hitachi drives really that much more expensive so that it doesn't justify their vastly longer lifespan? Even if they can get "free" replacement disks during the warranty period, that has a cost for them. And they mentioned that some replacement disks die even faster. I'm sure Backblaze has crunched all these numbers - would love to see them. BTW thanks for sharing this data!
- joeframbach 13y ago
  The cost you see on newegg probably isn't the cost Blackblaze sees. They're getting bulk pricing, which WD is probably more eager to sell than Hitachi.
  - brianwski 13y ago
    No, we are not getting bulk pricing. At least not really by much. We sometimes still "farm" where employees all visit Costco to buy drives RETAIL, we also buy from Amazon and NewEgg. Honestly, we just cannot seem to get good deals on drives! If anybody on earth can get these magically better prices, we are begging them to come forward, we promise to give you 99 percent of the cost savings. Beat our current suppliers by 1 penny per drive and you take all our business. We're buying about 800 drives per month, shut up and take our money! (Each drive is 4 TBytes)
- brianwski 13y ago
  Backblaze employee here - it is honestly just a spreadsheet that kicks out the answer. Every month we ask 20 or so suppliers for the lowest price for each drive type. If Hitachi are 10 percent more expensive but fail 10 percent less often, that balances out and we buy Hitachi. But if it is 12 percent more costly then we get the other brand. There is a tiny bit of free preference leeway given to Hitachi because it means less hassle to our over worked datacenter team...
  - olov 13y ago
    If I'm not reading it wrong then your data says that the Hitachi drives have half the Annual Failure Rate, or less, than the others (in your setup). Not sure what this means in MTBF but the Hitachi's sure seem to be worth a whole lot more, certainly 10, 20 or 30 percent more - no?
polskibus 13y ago
HGST (Hitachi) has been bought by Western Digital. One should be able to expect a merge of their HDD lines (source: http://www.hgst.com/ http://www.hgst.com/) . Moreover, it seems that Deskstars are no longer manufactured (or have been rebranded). http://www.hgst.com/hard-drives/product-brands http://www.hgst.com/hard-drives/product-brands
acd 13y ago
Also see hardware.fr failure rates. It shows different data than Backblaze. French hardware site, component failure rates. Google translate it to english http://www.hardware.fr/articles/911-6/disques-durs.html http://www.hardware.fr/articles/911-6/disques-durs.html So its also important to take hard drive models into account. Then there is the Google study Failure trends in large hard drive population http://static.googleusercontent.com/media/research.google.com/en//archive/disk_failures.pdf http://static.googleusercontent.com/media/research.google.co...
mlvljr 13y ago
small featherweight laptop Toshiba works as a main (and only) drive in the desktop box flawlessly -- no sound, no heat :)
desireco42 13y ago
Is this to counter bad pr from yesterday's story about that guy whose files you lost? Because it looks that way. No matter what hd you use, if you are corrupting files, it's all the same. Same with Evernote, if their sync is losing notes, and it is, everything else is less important.
- atYevP 13y ago
  Sean's issue was unfortunate. I think he posted the blog a little prematurely, our support is in contact with him and we're trying to figure out what happened. We restored a lot of data over Backblaze's life (over 5 billion files) and normally the .zip restores are rock solid, and we didn't have any outages yesterday. We're trying to collect his logs and see exactly what broke down. He said that he'd update his posts after the issue was resolved, so keep an eye on it!
  - desireco42 13y ago
    I will, but it cast serious shadow over your service :) however awesome other aspects are
    - atYevP 13y ago
      It happens. Computers are weird things, especially when networking is involved. We always recommend having a local and an off-site copy of data. That way you minimize risk, we're hoping we can get him back up and running soon!
      - GigabyteCoin 13y ago
        >Computers are weird things... That's not very encouraging, coming from a data storage company employee.
        memracom 13y ago
        It might not be encouraging but it is honest and it is factual. No company, however much you might wish it, can do magical things. At best, you can leverage the laws of large numbers to do things which are apparently magical to those people who do not have an understanding of maths, technology and engineering. This is exactly how stage magicians accomplish their tricks. A solid understanding of physics and high precision engineering.
deltaqueue 13y ago
From the article and throughout the comments here it seems Backblaze prefers cheaper drives over a few percentage points of reliability. It would be interesting to see some data showing the tradeoff, but I suspect it reveals too much of their operation. At first glance it appears you can get a drive with .9% failure rate (HGST 7K3000) for $127[1], and yet BB really likes the WD Red, which has a higher failure rate (3.2%) and cost[2]. What might shed light without revealing too much is information about where they source drives today (their sourcing coverage during the shortage was very cool!). I suspect they're finding some nice bulk discounts somewhere. [1] http://www.amazon.com/Hitachi-Deskstar-7K3000-HDS723030ALA640-SATA-600/dp/B004D8KO7Y http://www.amazon.com/Hitachi-Deskstar-7K3000-HDS723030ALA64... [2] http://www.amazon.com/WD-Red-NAS-Hard-Drive/dp/B008JJLW4M/ http://www.amazon.com/WD-Red-NAS-Hard-Drive/dp/B008JJLW4M/ (both seem to be market consumer prices)
- wting 13y ago
  If systems are designed with the expectation that hardware can and will fail often, better reliability drives aren't worth the cost as long as cheaper drives are relatively comparable. In addition to cost savings, your system has better robustness when it is decoupled from hardware reliability. For example, Google's Map Reduce paper has a section on fault tolerance that goes into detail about how they handle the issue of failing workers: http://research.google.com/archive/mapreduce.html http://research.google.com/archive/mapreduce.html
  - at-fates-hands 13y ago
    Are you taking into consideration when a drive fails it requires work to replace it? This could be minimal and something that in terms of budgetary considerations might be negligible - but I'm not sure.
    - brianwski 13y ago
      Backblaze employee here -> Yes, this gets a SMALL amount of allowance. The datacenter team begs us to buy the Hitachi drives even at twice the price, but it would bankrupt us. But if the Hitachis are only $2 or $3 more expensive per drive (including the failure rate in that calculation) then we're willing to buy them for the reduced hassle. I think the calculation is replacing one drive takes about 15 minutes of work. If we have 30,000 drives and 2 percent fail, it takes 150 hours to replace those. In other words, one employee for one month of 8 hour days. Getting the failure rate down to 1 percent means you save 2 weeks of employee salary - maybe $5,000 total? The 30,000 drives costs you $4 million, so who cares about $5k here or there?
wil421 13y ago
Neither you should buy an SSD.
- gwern 13y ago
  They're doing bulk storage where costs are a priority and performance irrelevant since users are uploading/downloading over the Internet. What possible use do they have for SSDs?
  - wil421 13y ago
    The title of the post is Which Hard Drive Should I Buy? Not which Hard Drive should Back Blaze buy again. As a consumer I wouldnt consider buying a HDD ever again. I would be more interested in a post that details the reliability of SSD manufactures/models.
    - darkarmani 13y ago
      Why would you need SSDs to backup your computer? Some people like to have a local backup and use the cloud for offsite backups, so they can recover much faster.
      - wil421 13y ago
        I probably wouldnt unless I have spares (which I do) but regardless of wether the SSD is used for backups or for a primary drive I would still like to see a list like this one.
    - Ecio78 13y ago
      There is still some people interested in HDD, for example those who hosts a n-Terabyte home NAS and that can't afford spending thousands of dollars[1] to fill it with SSD.. [1] http://www.zagg.com/community/blog/what-is-the-largest-ssd-solid-state-drive-available/ http://www.zagg.com/community/blog/what-is-the-largest-ssd-s...
      - 13y ago
Fomite 13y ago
This is a pretty textbook perfect application of survival/time-to-event analysis. Any chance the data behind it could be made available for teaching purposes?
- atYevP 13y ago
  Yev from Backblaze -> Where/what do you teach? We're unsure about releasing any more information at this time, but we're not opposed to it. What would it be used for?
  - adolgert 13y ago
    Maybe you looked at Schroeder and Gibson from Fast07, which looks at MTBF calculations. They have lots more data, but none released. Hard drive failures are an easy example of the use of the concept of hazard to examine probability distribution functions. I'm not teaching this now, but I sympathize with the difficulty of finding good datasets. Who knows? If you group data by pod, or even location in datacenter, there could be something interesting in exploratory analysis.
    - caycep 13y ago
      In my limited knowledge, this would be a generalized linear mixed model to look at factors influencing drive failure? or more of a poisson thing?
      - Fomite 13y ago
        You could use any number of models to model the time until drive failure - accelerated failure time models, a poisson model, a Cox proportional hazards model, Kaplan-Meier curves...
  - Fomite 13y ago
    I'm a postdoc at the moment, but I've taught survival analysis before. It just struck me as a particularly straightforward example with pretty clean data, a visible separation of the different groups, etc. Basically, lots of people use the "Iris" data set to learn either data visualization or machine learning based classification. The same could be true of the "Backblaze Hard Drive" data, but for survival analysis/time-to-event statistics courses.
Dystopian 13y ago
VERY interesting. I always avoided Hitachi drives in my deck for some reason - always concentrated on WD and Seagates (personal experience has led me to lean towards WD as well, as I had a bunch of those non-LP drives that they talk about that like to conk out). From what I read they look quite reliable over a fairly representative sample (annual failure rate v. # of drives / TBs / years).
- pixl97 13y ago
  > I always avoided Hitachi drives in my deck for some reason Probably because Hitachi bought IBM's line of Deathstar drives that were notoriously unreliable. It appears they have done far better with the product.
  - kalleboo 13y ago
    Yeah, I've always had a negative view of Hitachi since I had a lot of trouble with the Deathstar 75 GXPs. But Seagate bought Maxtor, and I never heard anything good about Maxtor aside from "they're the cheapest!", so I guess it goes both ways... I stopped hearing good things about Seagate right after they had to stop using their "silent" tech after some patent dispute, which IIRC coincided with the Maxtor acquisition...
  - brianwski 13y ago
    I've heard this rumor from everywhere, that the Deskstar drives were unreliable. If you look at the reviews from NewEgg they were TERRIBLE for that product. But I'm completely suspicious of where that reputation came from, because the statistics just don't bear it out. Often "common wisdom" is wrong - I suspect this is one of those cases.
    - pixl97 13y ago
      In 2000-2001 I was an admin for a day trading firm managing around 80 computers and servers. The drives worked great. They were really fast right up till the moment of their clicking death. At around 6 months around 15 out of 30 had it occur.
hackinthebochs 13y ago
Definitely wish I had seen this a couple months ago before I bought two 3TB seagates. Although to be fair I was already pretty sure that seagates sucked (its good to see data backing that up) but getting two for $85 each was too hard to pass up. I'm a sucker for a deal. I buy HDs in pairs now so I'm not too worried about losing anything. I am intrigued by backblaze's service though. A part of me feels like there must be a catch somewhere. I have a good 10TB I'd be happy to pay $5/month to backup but somehow I feel like they'd pull a comcast and say their "unlimited" claim doesn't apply to the 1% of users (or in this case maybe the .001%).
- stdbrouw 13y ago
  The catch is that the initial upload can take a while if your internet connection has limited upstream – not something Backblaze can help but I'm sure it has made some people think twice before having their external drives mirrored in the cloud.
  - brianwski 13y ago
    One idea that works for some people is to carry your entire computer to a "fast connection", for example your workplace. Plug it in, leave it running for 24 hours to power through the initial upload, then carry your computer home again for the incrementals. Just an idea.
    - arielweisberg 13y ago
      My experience has been that the upload speed is slow no matter what connection you are on. They claim not to throttle, but I am skeptical that you will get more than 1.5 megabit anywhere. I couldn't. They actually ship a bandwidth test that goes to their datacenters. If I ran multiple instances of the test at once I got 1.5 megabits for each instance. It's not a bandwidth or capacity issue... If they documented 1.5 megabits I wouldn't complain, but claiming one thing and shipping a product that throttles, whether explicitly or implicitly, is obnoxious. I did a restore recently and was pleasantly surprised to get 9 megabytes/sec down so that is much better and restore speed is a higher priority. However if I had to upload the entire data set from scratch I would stop using Backblaze.
cocoflunchy 13y ago
How does that compare to SSD failure rates? Are they much better?
- KamiCrit 13y ago
  I dunno if SSDs have been around long enough for reliability testing. Along with they have been advancing quite rapidly also so it is hard to stick with something and expect the info to be relevant down the road. All I know when it comes to SSD reliability, Intel is king.
protomyth 13y ago
In the last two year, I have stopped all buying of Seagate drives. We had a rather large rash of failures in RAIDs and desktops (50% of about 50 drives). Then the "every drive shipped by HP is failing" problems of the netbooks (70 total replace 50 drives) were also Seagate drives. We are basically an WD house now.
justin66 13y ago
> The drives that just don’t work in our environment are Western Digital Green 3TB drives and Seagate LP (low power) 2TB drives. Both of these drives start accumulating errors as soon as they are put into production. We think this is related to vibration. The drives do somewhat better in the new low-vibration Backblaze Storage Pod, but still not well enough. Another reason to avoid the WD Green 3TB: these drives aggressively put themselves to sleep to save power. It's literally a matter of streaming a video from disk and if the OS caches enough of the file, the drive will see there haven't been any accesses in a few seconds and stop spinning. The video will of course glitch when the cached data runs out and the drive needs to spin up. Great design.
- B-Con 13y ago
  Don't use the Green line for anything performance related, even if it's watching videos. The Green line is simply not intended for that. I think it's designed to be an occasionally-accessed backup drive, or something like that. The Black line is the performance-conscious line, look at the corresponding equivalents in that line. IIRC the Blacks are 10-15% more expensive, but they actually respond.
  - IgorPartola 13y ago
    I recently bought two Green drives for my NAS not knowing the difference. Do your research :). Having said that, the NAS is working out pretty well so far with ZFS running on top of them.
    - frenchman_in_ny 13y ago
      I bought 3 Green 2.5TB drives for a Synology NAS that's lightly used -- and all 3 have had SMART failures in the ~18 months that I've owned them. WD just sent me a refurb'ed Green 3TB drive, let's see if it holds out better. I wish I had had access to this data before buying the Greens...
    - dpe82 13y ago
      I've had 4 Green 3TB drives in a raidZ for less than a year. First few months were great, then I had to replace two of them a couple months ago due to data errors and another just started failing so today (literally as I write this) I'm replacing the other two. Don't use Greens for anything that's on and could be active 24/7. The aggressive load cycling kills them.
undoware 13y ago
Aside from the incredible usefulness of the data herein -- thanks, backblaze! -- this is also the kind of marketing that I don't mind. Backblaze got egg on its face yesterday on HN when someone's critical report (rightly) got upvoted. Today they make up for it by giving us an interesting and useful data chart. I know it sounds weird, but for whatever reason I read this as demonstrating a high level of corporate responsibility and attunement to customers. They could have compensated by instead, say, dropping a few grand on buying journalists and 'reviewers', like Microsoft does. But they didn't. Instead, they were cool. To me that signals that they'll also take care of whatever problems they've had recently. (Note: I have no affiliation with Backblaze.) I'm currently not shopping for an online backup service but if that ever changes, I now have a good feeling about Backblaze, and I hope that other services take a similar approach to repairing customer relations when they are fraught. Shit happens, even to backup providers. It's how you respond that matters most.
lae 13y ago
If only I could find the Hitachi Deskstars for cheap.
cordite 13y ago
Interesting note on the WD 3TB Green. I have one in my rig, and whenever I do any disk access, I always have to wait about 5-8 seconds for it to spin up every other half hour. It seems to aggressively turn off. I have my base system on an SSD, and my games and other things on this 3TB drive. I guess such spin up times would be unacceptable in backblaze's environment.
rythie 13y ago
I wish Backblaze would provide some sort of Amazon S3 competitor, Amazon always seems very overpriced.
- atYevP 13y ago
  Hypothetically, what would you be looking for in an S3 competitor? Specifically: 1. How much data do you have? 2. What would you use it for? 3. How important is it if the API isn't the same as the S3 API? 4. Any specific certification requirements? 5. Would you have any SLA requirements? 6. Specific performance metrics? Think Amazon S3 vs. Glacier. 7. Are redundant data centers important? Hypothetically!
  - rythie 13y ago
    I think the main problem is cost, Amazon's costs prevent startups like Everpix from existing as does it for video startups and various other startups that require a lot of storage. I think Digital Ocean's disruption of the very established VPS market is the model to follow. Personally I'd like to have all my data in the cloud. 1. 300GB growing at 25-50GB/year 2. Backup / cloud storage 3. Possibly, though I'd like SFTP access really 4. no 5. Don't lose my data, probably not down for more that 3-4 hours in one go. 6. Enough to stream video to back to my box 7. Not really, I'd assume the chance of data center being down is pretty small. Hitachi 500GB USB $2.4/month (based on $60 drive with 2 year warranty) Backblaze $5/month [Unlimited] Crashplan $5/month [Unlimited] Amazon charge $22.80/month Dropbox $50/month [500GB] (though I still need a local copy) Digital Ocean instance $320/month I think there is opportunity to come in at half of Amazon's price or less and that could lead to a new set of start-ups that could build on that.
    - atYevP 13y ago
      Thanks for taking the time to answer! Interesting use-case!
happycube 13y ago
Two things I never like to see together on a hard drive: "Seagate" and "Made in China"
ck2 13y ago
I'm a big fan of single platter drives, I just buy the biggest single platter at the time. Which is currently the WD 1gb blue. Very fast, very cool running.
- krakensden 13y ago
  1GB? that seems... awful small.
  - scholia 13y ago
    Massive compared to the 10MB drive in the IBM PC/XT....
  - baddox 13y ago
    And even 1TB, which is presumably what he meant, feels very small to me.
- PavlovsCat 13y ago
  Any particular advantages of having a single platter? Also, I assume you meant to say 1TB ^^
  - grey413 13y ago
    The logic is that fewer moving parts means a lower failure rate. It amounts to paying a premium for longer-lived drives. How much you value that is a matter of taste and means.
- paol 13y ago
  The fewer platters the better for vibration and noise as well. Though I wouldn't necessary restrict myself to single-platter drives, 2 platters is generally fine.
- theandrewbailey 13y ago
  It's really unfortunate that WD has completely halted expanding capacity on their Blue line. Meanwhile, my WD6400AAKS (640GB blue, 2 platter) keeps marching on.
deleted 13y ago
[deleted]
goofygrin 13y ago
Seagate Barracuda Green (ST1500DL003) 1.5TB 51 0.8 120.0% I've got one sitting on my desk that seagate sent me as a RMA return. Guess it won't be going back into our RAID.
cbr 13y ago
Any stats on power consumption? Over 5 years the difference between a drive that uses 6 Watts and one that uses 7 is 44kWh or about $5. Double that to include cooling costs and saving a Watt should be worth something like $10 to you, so a more expensive more efficient drive could be worth it. Do these drives all use similar amounts of power?
- ars 13y ago
  Cooling costs are an extra one third (and even that is only during the summer), not even close to double.
  - gibybo 13y ago
    Do you have a source or at least an explanation for that? I've always thought it took at least as much energy to cool something as it did to heat it (because AC is not 100% efficient), but I've been pretty wrong about thermodynamics in the past so I'd like to dive a bit deeper here. Maybe this is only true for AC, and not water cooling being pumped to passive radiators on the roof? EDIT: Here: https://www.google.com/about/datacenters/efficiency/internal/ https://www.google.com/about/datacenters/efficiency/internal..., Google is claiming a 'Power usage effectiveness' of 1.11 across their data centers. This implies a cooling cost <= 11% of their server costs which I found quite surprising.
    - TheCoelacanth 13y ago
      Hard drives are going to run hotter than the outside temperature even in the middle of the summer, so they are going to get quite a bit of natural cooling for free. The cooling they have to do is just a supplement to the cooling that happens naturally.
    - ars 13y ago
      It's pretty easy to calculate. Just find a window A/C and check the BTU it's rated for and the wattage. Convert BTU to watts, then do wattage to power the A/C divided by the watts of cooling power. Then note that this is the worse case scenario - normally it does better than that except on the very hottest days of the year.
halayli 13y ago
Lesson from this: buy yourself a hitachi.
caycep 13y ago
I'm just an amateur at statistics, but I would think this would be a useful set to do generalized linear mixed models on, to see what factors could be statistically significant (manufacturer, model, factory location, etc etc etc)
ChuckMcM 13y ago
Great post, as you get bigger populations of drives you can get a lot more visibility into their overall reliability. If there was one thing I could add to the analysis would be to split out the drives by serial number and split them out by firmware. Sometimes you find that all of the 'problem' in a set of problem drives is a single range of serial numbers. We've had similar experiences with replacement drives, they are, by and large, significantly less reliable than "new" drives. And last bit, we've got Western Digital drives here (a mix of 2.0 and 3.0 TB ones) They have been pretty solid performers for us.
bovermyer 13y ago
I am very impressed by the transparency here, and appreciative of the data. I'd never heard of Backblaze until now, but now I'll have to pay closer attention.
jpalomaki 13y ago
Posts like this are the reason why I trust Backblaze. In general I'm very suspicious about the unlimited offerings. When they tell the technical details I get the feeling they are up to task and not just reselling S3 and hoping people pay but don't actually use the storage.
jader201 13y ago
> We are focusing on 4TB drives for new pods. For these, our current favorite is the Seagate Desktop HDD.15 (ST4000DM000). We’ll have to keep an eye on them, though. Historically, Seagate drives have performed well at first, and then had higher failure rates later. I'm a little surprised that they actually did the analysis to determine the Seagates tend to fail more, yet they are still putting most (or at least, quite a bit) of their faith in those. Based on their own data, I would likely avoid those, or at least start leaning more toward Hitachi and WD. Or maybe the initial cost of those is so much better that it compensates for any long-term expense.
- gatehouse 13y ago
  You also have to consider that when the drives eventually fail, they will be replaced with hard drives of the future -- which will presumably be cheaper than the HDD of today. I.e. they depreciate quickly.
InclinedPlane 13y ago
Back in 2007/2008 or so I bought a pair of Seagate 7200 500gb drives and got bit hard by their extremely broken firmware. I haven't bought seagate drives since then, it's sad that their overall QC is still so terrible across the board.
madads 13y ago
Should consider what Jacob Applebaum revealed in his CCC preso about Hard drives: http://youtu.be/vILAlhwUgIU?t=46m25s http://youtu.be/vILAlhwUgIU?t=46m25s Cross out WD, Seagate, Maxtor and Samsung drives. Hitachi wins!
gnoway 13y ago
As an administrator with an admittedly smaller sample size, this lines up with my experience with the Seagate 1TB Barracuda ES.2, 15K.6/15K.7-era Cheetahs and some normal consumer Barracudas as well. All of the nearline and enterprise stuff had 5 year warranties, and over the past 5 years we've replaced over 25%. I've sworn off Seagate altogether, at least until they demonstrate a commitment to producing more reliable drives. I will not willingly buy them for the datacenter, and I won't buy them at any price for the home, RAID or no RAID.
dshep 13y ago
Dang! I just bought WD Green 2TB WD20EZRX drives (x4) for my NAS.
edward 13y ago
Price per TB: http://edwardbetts.com/price_per_tb/ http://edwardbetts.com/price_per_tb/
benrapscallion 13y ago
Along these lines, I have done a monetary analysis; http://perenniallycurious.com/centspergb.html http://perenniallycurious.com/centspergb.html
chemmail 13y ago
Shouldn't Toshiba be the drives to buy since they took over Hitachi's 3.5 drive factories when WD took over due to antitrust? http://www.wdc.com/en/company/pressroom/releases/?release=f8697da0-111a-49e8-a484-5266d18526ab http://www.wdc.com/en/company/pressroom/releases/?release=f8...
csense 13y ago
A better presentation of this data would show a failure rate for each brand and month/year of purchase. For extremely simple devices like resistors or incandescent light bulbs, failure rate is relatively constant over the lifetime of device -- the chance of a functioning resistor with 10 hours of use failing during the next hour is the same as the chance of a functioning resistor with 1000 hours of use. For complex devices with lots of interdependent parts, some of which are mechanical, the failure rate changes over time. There's an "infant mortality" or "lemon" phenomenon, where relatively new devices have higher defect rates (because fabrication and shipping sometimes result in imperfections which quickly cause failures), followed by a steep dropoff in failure rates (because observing a device operate correctly for dozens of hours is strong evidence that it doesn't suffer from a failure mode which often results in infant mortality). Then there may be an increase in failure rates later, especially with devices that are partially or wholly mechanical (wear or damage type problems which do not cause immediate failure, but make it easier for a failure to occur). You need empirical data to be quantitative about this curve, and it sounds like Backblaze has it, but their presentation in this article doesn't show it.
- hga 13y ago
  As I recall one of the studies of a few years ago, the one based on supercomputers, not Google's, showed there was very little infant mortality, and wear clearly set in after roughly one year in service. The results were quite striking, and nothing like the bathtub curve many expected and that you sort of sketch out.
abcd_f 13y ago
Interesting stats, but I wonder if the drive usage pattern at BackBlaze is somewhat different from that of a home user. In terms of how they seek, read and write. Not that they put more mileage on them faster, but that they might be doing something fundamentally different from how these drives are tested by their manufacturer. Lots of bulk in-sequence writes or something else. The 5-7 times difference in failure rates between leading manufacturers is frankly hard to believe.

688 points by nuriaion 13y ago

aroch 13y ago

Heh...I, too, love 3TB WD REDs. At last count I have 40 of these racked up in servers or NASes and they've worked quite well for me

mustafab 13y ago

Too bad you still don't have linux client. Do you think supporting linux users anytime soon?

Nickoladze 13y ago
Relevance?
- blisterpeanuts 13y ago
  Well, it's off the subject, but it's still an interesting question and I was wondering the same thing.

dnissley 13y ago

Interesting. I had always avoided Hitachi Deskstars after having heard they were nicknamed "Deathstars" for a reason. Perhaps that was once true, but clearly it's not anymore.

michaelt 13y ago
Apparently that was 2001 and they were IBM Deskstars at the time [1]. I guess this is why companies sometimes decide to rebrand/rename products. [1] https://en.wikipedia.org/wiki/HGST_Deskstar#IBM_Deskstar_GXP-model_failures_in_2001 https://en.wikipedia.org/wiki/HGST_Deskstar#IBM_Deskstar_GXP...
yen223 13y ago
I suspect those infamous disk failures were caused by faulty manufacturing runs, rather than some inherent flaw in the design.
- radiowave 13y ago
  IIRC, it was specific to the 38GB model, and was traced to a batch of bad electronic components.
  - nknighthb 13y ago
    No, it was the whole 75GXP set (actually, according to Wikipedia, the 120GXP and 180GXP were also affected to a far lesser extent). The backlash was really nasty, though, because IBM's drives were something a lot of us had come to trust and rely on, the 75GXP was kind of a flagship drive, and IBM's handling of the issue was less than stellar. We felt betrayed. I never have bought another IBM (or Hitachi) drive. The reaction is visceral and intractable.
    - radiowave 13y ago
      Ah yes. I think I had the jumper on mine set to limit the drive to 38GB. Luckily, mine was OK.
- justin66 13y ago
  I bought an IBM Deskstar and went through five returns over the course of a couple of years - across two different generations of drives! (The 75GXP and the 60GXP, I believe) I will say their warranty service was generally not bad... they seemed to get a lot of practice. The interesting thing is that what you say strikes me as almost certainly true, in a narrow way: there were probably some manufacturing runs that were a lot better than others. The proportion of bad runs to good runs is significant.
- BashiBazouk 13y ago
  I worked in platter sputtering at IBM right out of College during the Death Star period. Basically, IBM had multiple parts manufacturing and sputtering facilities all over the world. The best disks/heads/motors always went in to the SCSI drives and the lesser parts went in to IDE drives. SCSI drives of that period were fairly bullet proof. Desktop IDE drives, not so much. Most of the San Jose and Munich production had higher yields and were mainly for SCSI and laptop drives. So it was less faulty manufacturing runs than the parts came from lesser production facilities, usually Singapore and the Philippines. They ended up closing the sputtering plant in the Philippines as they were never able to get their test yields up high enough...
wmf 13y ago
If you avoid Hitachi due to that bad batch and Seagate due to their bad batch and WD just because, you're left with nothing.
- hga 13y ago
  To my understanding (see my other comment in this subthread for a bit more), neither Seagate nor WD in at least recent times (21st Century?) screwed up even close to that badly.
hga 13y ago
Well, there was a period where a company that failed that badly apparently lost so much business they were forced to sell it or the division. IBM Deathstars (which I missed buying my the most minuscule fractions time, phew) resulting a sale to Hitachi, Maxtor I gather to Seagate.

staticshock 13y ago

Interesting to see that kind of a difference between hitachi and western digital, given that WD owns HGST. Are hitachi drives marketed as higher reliability drives, or was the acquisition by WD simply too recent for the quality of the two brands to "equalize"?

davis_m 13y ago
Backblaze's data shows that the number of errors is largely related to the age of the drive. Because the older drives are from before Hitachi was acquired by WD, it is going to take a few more years for the brands to equalize if they do combine the manufacturing of both lines.

nwmcsween 13y ago

The worst service and hard drives I have ever had were from seagate. I had 3 separate hard drives RMA'ed _twice_ and all three failed within 3 months twice over, to this day I will not buy seagate.

DanBC 13y ago

Do the Thailand floods make any difference to this report? How reliable were those drives, and are the factories ba k up to full speed yet?

yen223 13y ago
The factories have been at full speed for the past year now.
- pstack 13y ago
  You wouldn't know it by the prices.
  - izzydata 13y ago
    The prices are pretty much where they were from before the flood. Harddrive capacity has just gone up a ton. I bought a 1 terabyte before the flood for $60 and that is pretty much where it is at now if not cheaper in some cases.
    - gwern 13y ago
      Yes, but the problem is, hard drive $/GB has not recovered to the pre-flood trendline, even if prices have (4 years later) finally hit parity. The trendline has been badly hurt by the floods, it's pretty amazing. (I did a little tracking and was impressed how my predictions of the consequences were still insufficiently pessimistic: http://www.gwern.net/Slowing%20Moore%27s%20Law#kryders-law http://www.gwern.net/Slowing%20Moore%27s%20Law#kryders-law )
      - muyuu 13y ago
        Economies of scale haven't recovered to pre-flood levels, because of the crisis in developed countries and because SSD is quickly growing its share of the market. These are completely different technologies and their R&D spending and amortisation is separate.

freshyill 13y ago

I wonder if any of this actually applies to consumer-grade drives. My wife's hard drive actually just died. It was 160 GB WD in a black 2006 MacBook. The drive itself was a replacement from 2007 since the original drive died just over a year into its life. Stupidly, since her Time Machine backup was misbehaving, I reformatted it and set it to start over. I spent the weekend recovering her data—with a lot of success, so no big deal. At any rate, this machine is long past its expiration date. It's time for a MacBook Air with an SSD, once the tax refund comes in.

atYevP 13y ago
Yev from Backblaze here -> All of our drives are consumer-grade. We try to avoid buying enterprise drives at all costs. These are all off-the-shelf internals and in some cases...externals that were made internal! :)
- sehrope 13y ago
  > and in some cases...externals that were made internal! :) I remember reading about this a while back[1] (fun read!). Have you guys noticed a big difference in life expectancy of the repurposed external drives or do they generally match up with their internal equivalents? [1]: http://blog.backblaze.com/2012/10/09/backblaze_drive_farming/ http://blog.backblaze.com/2012/10/09/backblaze_drive_farming...
  - atYevP 13y ago
    Interestingly, we think they do as well if not better (http://blog.backblaze.com/2013/12/04/enterprise-drive-reliability/ http://blog.backblaze.com/2013/12/04/enterprise-drive-reliab...). We think it's because external drives are meant to be jostled around by Joe Schmo, so they do well in our storage pods, whereas internal drives and especially enterprise drives are meant to live in a perfectly Utopian environment.
- lsc 13y ago
  Have you seen a different failure rate between the drives that were sold as internal vs. the drives that were sold as external and "shucked" ?

kylec 13y ago

This makes me feel better about the 4 HGST 4TB drives I just bought (http://www.newegg.com/Product/Product.aspx?Item=N82E16822145912 http://www.newegg.com/Product/Product.aspx?Item=N82E16822145...). They were the cheapest 4TB 7200RPM drives on Newegg by a non-trivial margin.

atYevP 13y ago
<3 <3 <3 <3 <3 Hitachi drives. If we (Backblaze) could get them at a rate within a few dollars of the other manufacturers at the 4TB level, we would solely buy them.
- reitzensteinm 13y ago
  Surely not? If a specific model happens to have a large defect rate, having all your eggs in one basket could prove absolutely disastrous. Spreading your drives out, even among inferior options, seems like the only solid strategy.
  - atYevP 13y ago
    Normally diversification is absolutely the way to go, but once we've found a drive with low failure rates across the board, we'd want to move mostly to using that one drive type. We can always move over to another type of drive, but there's a very real cost of having to swap drives out as they fail, so the more reliable the drive, the more incentive to stick with it until it's no longer reliable. At the moment, we're more concerned about price, so we buy a wide-variety, but we have our favorites :)
    - scholia 13y ago
      If you're buying something to run for 5 years x 365 days x 24 hours, then how big a percentage of bucks is worth saving?
    - justin66 13y ago
      > Normally diversification is absolutely the way to go, but once we've found a drive with low failure rates across the board, we'd want to move mostly to using that one drive type. With regard to deployment, but also with regard to the drive reliability numbers you guys are putting out there: do you worry that variation between manufacturing runs is going to hose you? Do you find looking at the drives that you're getting a good variety of hardware from multiple manufacturing runs?

brokentone 13y ago

Very cool that Backblaze continues to post things like this. Few people have this experience. Of those who do, few (I assume) break it out to this level of detail, actionable for others. Of those who have good experience and records, most would consider it proprietary or just decide not to post. Kudos to Backblaze.

atYevP 13y ago
We're all on this crazy spinning ball together. Early on Backblaze made the decision to stay out of the hardware business (it sounded lucrative after we developed the v1 storage pod, but everyone was a software person, so we went that route), we like to pay our hardware findings forward. It's a good thing to do!
- nickff 13y ago
  I really appreciate this; as someone who has had a number of SSD and HDD failures due to lackluster manufacturers (OCZ and Seagate respectively).
- Zancarius 13y ago
  I really like this detailed write up. A certain large company in Mountain View suggested years ago that they had collected a great deal of metrics on consumer grade hard disks performing in a data center capacity, but refused to publish the data. The fact that Backblaze isn't embarrassed to share this with the rest of this gives me the warm fuzzies. Thanks for the hard work! I recently purchased a new Seagate, and I'm feeling a bit of buyer's remorse. That's not to say this article triggered the feeling completely (though it did push me over the edge). Taking this information and the knowledge that recent model Barracudas appear to have aggressive APM that loads/unloads the heads after a minute or so of inactivity (unless disabled, but it doesn't persist between reboots) AND seems to borrow from laptop-grade design... I think I'd be better off replacing it. Again. ;) I wonder if the poor longevity of Seagate drives is due to changes in quality control or design? I have older ones that don't utilize the APM load/unload features and are still running fine after many years. Then again, they were also pre-flood drives. I'd also be curious to know if there's a correlation with where the drive was manufactured and its survival rate, though I suspect that might be a much more difficult metric to collect.

izzydata 13y ago

Should the larger capacity seagate drives have a higher failure rate than small ones? That chart seems counter intuitive.

unreal37 13y ago
They said in the article that the 1.5TB drives were largely warranty replacements, and they suspect they were refurbs.

wnevets 13y ago

the WD reds only have 3 stars on newegg.com, interesting.

zanny 13y ago
Thing is most reviews are written immediately after recieving, and I'd rather get a defective drive in the mail and just RMA it on day 1 with the confidence when I get a working drive it will last longer than to have a drive that always works flawlessly out of the box (some 3tb seagate model I know got like 4.5 stars) but usually fails in a year. Though my current storage drive is a 3tb seagate one, that has been working fine for 2 years with no smart errors. Problem is I need to get the money to build a more appropriate backup raid for all my music / movies / games / etc, but that would be a pretty penny. I've just been using old dvd-rs (I mean, who uses those anymore?) as an impasse with anything really important on my 1TB external disks.
whichdan 13y ago
They have great reviews on Amazon, though.
insertnickname 13y ago
Also, the table in the blog post says their average life is 0.5 years... Maybe I misunderstood something.

olov 13y ago

"If the price were right, we would be buying nothing but Hitachi drives." I don't understand why they don't. Are the Hitachi drives really that much more expensive so that it doesn't justify their vastly longer lifespan? Even if they can get "free" replacement disks during the warranty period, that has a cost for them. And they mentioned that some replacement disks die even faster. I'm sure Backblaze has crunched all these numbers - would love to see them. BTW thanks for sharing this data!

joeframbach 13y ago
The cost you see on newegg probably isn't the cost Blackblaze sees. They're getting bulk pricing, which WD is probably more eager to sell than Hitachi.
- brianwski 13y ago
  No, we are not getting bulk pricing. At least not really by much. We sometimes still "farm" where employees all visit Costco to buy drives RETAIL, we also buy from Amazon and NewEgg. Honestly, we just cannot seem to get good deals on drives! If anybody on earth can get these magically better prices, we are begging them to come forward, we promise to give you 99 percent of the cost savings. Beat our current suppliers by 1 penny per drive and you take all our business. We're buying about 800 drives per month, shut up and take our money! (Each drive is 4 TBytes)
brianwski 13y ago
Backblaze employee here - it is honestly just a spreadsheet that kicks out the answer. Every month we ask 20 or so suppliers for the lowest price for each drive type. If Hitachi are 10 percent more expensive but fail 10 percent less often, that balances out and we buy Hitachi. But if it is 12 percent more costly then we get the other brand. There is a tiny bit of free preference leeway given to Hitachi because it means less hassle to our over worked datacenter team...
- olov 13y ago
  If I'm not reading it wrong then your data says that the Hitachi drives have half the Annual Failure Rate, or less, than the others (in your setup). Not sure what this means in MTBF but the Hitachi's sure seem to be worth a whole lot more, certainly 10, 20 or 30 percent more - no?

polskibus 13y ago

HGST (Hitachi) has been bought by Western Digital. One should be able to expect a merge of their HDD lines (source: http://www.hgst.com/ http://www.hgst.com/) . Moreover, it seems that Deskstars are no longer manufactured (or have been rebranded). http://www.hgst.com/hard-drives/product-brands http://www.hgst.com/hard-drives/product-brands

acd 13y ago

Also see hardware.fr failure rates. It shows different data than Backblaze. French hardware site, component failure rates. Google translate it to english http://www.hardware.fr/articles/911-6/disques-durs.html http://www.hardware.fr/articles/911-6/disques-durs.html So its also important to take hard drive models into account. Then there is the Google study Failure trends in large hard drive population http://static.googleusercontent.com/media/research.google.com/en//archive/disk_failures.pdf http://static.googleusercontent.com/media/research.google.co...

mlvljr 13y ago

small featherweight laptop Toshiba works as a main (and only) drive in the desktop box flawlessly -- no sound, no heat :)

desireco42 13y ago

Is this to counter bad pr from yesterday's story about that guy whose files you lost? Because it looks that way. No matter what hd you use, if you are corrupting files, it's all the same. Same with Evernote, if their sync is losing notes, and it is, everything else is less important.

atYevP 13y ago
Sean's issue was unfortunate. I think he posted the blog a little prematurely, our support is in contact with him and we're trying to figure out what happened. We restored a lot of data over Backblaze's life (over 5 billion files) and normally the .zip restores are rock solid, and we didn't have any outages yesterday. We're trying to collect his logs and see exactly what broke down. He said that he'd update his posts after the issue was resolved, so keep an eye on it!
- desireco42 13y ago
  I will, but it cast serious shadow over your service :) however awesome other aspects are
  - atYevP 13y ago
    It happens. Computers are weird things, especially when networking is involved. We always recommend having a local and an off-site copy of data. That way you minimize risk, we're hoping we can get him back up and running soon!
    - GigabyteCoin 13y ago
      >Computers are weird things... That's not very encouraging, coming from a data storage company employee.
      - memracom 13y ago
        It might not be encouraging but it is honest and it is factual. No company, however much you might wish it, can do magical things. At best, you can leverage the laws of large numbers to do things which are apparently magical to those people who do not have an understanding of maths, technology and engineering. This is exactly how stage magicians accomplish their tricks. A solid understanding of physics and high precision engineering.

deltaqueue 13y ago

From the article and throughout the comments here it seems Backblaze prefers cheaper drives over a few percentage points of reliability. It would be interesting to see some data showing the tradeoff, but I suspect it reveals too much of their operation. At first glance it appears you can get a drive with .9% failure rate (HGST 7K3000) for $127[1], and yet BB really likes the WD Red, which has a higher failure rate (3.2%) and cost[2]. What might shed light without revealing too much is information about where they source drives today (their sourcing coverage during the shortage was very cool!). I suspect they're finding some nice bulk discounts somewhere. [1] http://www.amazon.com/Hitachi-Deskstar-7K3000-HDS723030ALA640-SATA-600/dp/B004D8KO7Y http://www.amazon.com/Hitachi-Deskstar-7K3000-HDS723030ALA64... [2] http://www.amazon.com/WD-Red-NAS-Hard-Drive/dp/B008JJLW4M/ http://www.amazon.com/WD-Red-NAS-Hard-Drive/dp/B008JJLW4M/ (both seem to be market consumer prices)

wting 13y ago
If systems are designed with the expectation that hardware can and will fail often, better reliability drives aren't worth the cost as long as cheaper drives are relatively comparable. In addition to cost savings, your system has better robustness when it is decoupled from hardware reliability. For example, Google's Map Reduce paper has a section on fault tolerance that goes into detail about how they handle the issue of failing workers: http://research.google.com/archive/mapreduce.html http://research.google.com/archive/mapreduce.html
- at-fates-hands 13y ago
  Are you taking into consideration when a drive fails it requires work to replace it? This could be minimal and something that in terms of budgetary considerations might be negligible - but I'm not sure.
  - brianwski 13y ago
    Backblaze employee here -> Yes, this gets a SMALL amount of allowance. The datacenter team begs us to buy the Hitachi drives even at twice the price, but it would bankrupt us. But if the Hitachis are only $2 or $3 more expensive per drive (including the failure rate in that calculation) then we're willing to buy them for the reduced hassle. I think the calculation is replacing one drive takes about 15 minutes of work. If we have 30,000 drives and 2 percent fail, it takes 150 hours to replace those. In other words, one employee for one month of 8 hour days. Getting the failure rate down to 1 percent means you save 2 weeks of employee salary - maybe $5,000 total? The 30,000 drives costs you $4 million, so who cares about $5k here or there?

wil421 13y ago

Neither you should buy an SSD.

gwern 13y ago
They're doing bulk storage where costs are a priority and performance irrelevant since users are uploading/downloading over the Internet. What possible use do they have for SSDs?
- wil421 13y ago
  The title of the post is Which Hard Drive Should I Buy? Not which Hard Drive should Back Blaze buy again. As a consumer I wouldnt consider buying a HDD ever again. I would be more interested in a post that details the reliability of SSD manufactures/models.
  - darkarmani 13y ago
    Why would you need SSDs to backup your computer? Some people like to have a local backup and use the cloud for offsite backups, so they can recover much faster.
    - wil421 13y ago
      I probably wouldnt unless I have spares (which I do) but regardless of wether the SSD is used for backups or for a primary drive I would still like to see a list like this one.
  - Ecio78 13y ago
    There is still some people interested in HDD, for example those who hosts a n-Terabyte home NAS and that can't afford spending thousands of dollars[1] to fill it with SSD.. [1] http://www.zagg.com/community/blog/what-is-the-largest-ssd-solid-state-drive-available/ http://www.zagg.com/community/blog/what-is-the-largest-ssd-s...
    - 13y ago

Fomite 13y ago

This is a pretty textbook perfect application of survival/time-to-event analysis. Any chance the data behind it could be made available for teaching purposes?

atYevP 13y ago
Yev from Backblaze -> Where/what do you teach? We're unsure about releasing any more information at this time, but we're not opposed to it. What would it be used for?
- adolgert 13y ago
  Maybe you looked at Schroeder and Gibson from Fast07, which looks at MTBF calculations. They have lots more data, but none released. Hard drive failures are an easy example of the use of the concept of hazard to examine probability distribution functions. I'm not teaching this now, but I sympathize with the difficulty of finding good datasets. Who knows? If you group data by pod, or even location in datacenter, there could be something interesting in exploratory analysis.
  - caycep 13y ago
    In my limited knowledge, this would be a generalized linear mixed model to look at factors influencing drive failure? or more of a poisson thing?
    - Fomite 13y ago
      You could use any number of models to model the time until drive failure - accelerated failure time models, a poisson model, a Cox proportional hazards model, Kaplan-Meier curves...
- Fomite 13y ago
  I'm a postdoc at the moment, but I've taught survival analysis before. It just struck me as a particularly straightforward example with pretty clean data, a visible separation of the different groups, etc. Basically, lots of people use the "Iris" data set to learn either data visualization or machine learning based classification. The same could be true of the "Backblaze Hard Drive" data, but for survival analysis/time-to-event statistics courses.

Dystopian 13y ago

VERY interesting. I always avoided Hitachi drives in my deck for some reason - always concentrated on WD and Seagates (personal experience has led me to lean towards WD as well, as I had a bunch of those non-LP drives that they talk about that like to conk out). From what I read they look quite reliable over a fairly representative sample (annual failure rate v. # of drives / TBs / years).

pixl97 13y ago
> I always avoided Hitachi drives in my deck for some reason Probably because Hitachi bought IBM's line of Deathstar drives that were notoriously unreliable. It appears they have done far better with the product.
- kalleboo 13y ago
  Yeah, I've always had a negative view of Hitachi since I had a lot of trouble with the Deathstar 75 GXPs. But Seagate bought Maxtor, and I never heard anything good about Maxtor aside from "they're the cheapest!", so I guess it goes both ways... I stopped hearing good things about Seagate right after they had to stop using their "silent" tech after some patent dispute, which IIRC coincided with the Maxtor acquisition...
- brianwski 13y ago
  I've heard this rumor from everywhere, that the Deskstar drives were unreliable. If you look at the reviews from NewEgg they were TERRIBLE for that product. But I'm completely suspicious of where that reputation came from, because the statistics just don't bear it out. Often "common wisdom" is wrong - I suspect this is one of those cases.
  - pixl97 13y ago
    In 2000-2001 I was an admin for a day trading firm managing around 80 computers and servers. The drives worked great. They were really fast right up till the moment of their clicking death. At around 6 months around 15 out of 30 had it occur.

hackinthebochs 13y ago

Definitely wish I had seen this a couple months ago before I bought two 3TB seagates. Although to be fair I was already pretty sure that seagates sucked (its good to see data backing that up) but getting two for $85 each was too hard to pass up. I'm a sucker for a deal. I buy HDs in pairs now so I'm not too worried about losing anything. I am intrigued by backblaze's service though. A part of me feels like there must be a catch somewhere. I have a good 10TB I'd be happy to pay $5/month to backup but somehow I feel like they'd pull a comcast and say their "unlimited" claim doesn't apply to the 1% of users (or in this case maybe the .001%).

stdbrouw 13y ago
The catch is that the initial upload can take a while if your internet connection has limited upstream – not something Backblaze can help but I'm sure it has made some people think twice before having their external drives mirrored in the cloud.
- brianwski 13y ago
  One idea that works for some people is to carry your entire computer to a "fast connection", for example your workplace. Plug it in, leave it running for 24 hours to power through the initial upload, then carry your computer home again for the incrementals. Just an idea.
  - arielweisberg 13y ago
    My experience has been that the upload speed is slow no matter what connection you are on. They claim not to throttle, but I am skeptical that you will get more than 1.5 megabit anywhere. I couldn't. They actually ship a bandwidth test that goes to their datacenters. If I ran multiple instances of the test at once I got 1.5 megabits for each instance. It's not a bandwidth or capacity issue... If they documented 1.5 megabits I wouldn't complain, but claiming one thing and shipping a product that throttles, whether explicitly or implicitly, is obnoxious. I did a restore recently and was pleasantly surprised to get 9 megabytes/sec down so that is much better and restore speed is a higher priority. However if I had to upload the entire data set from scratch I would stop using Backblaze.

cocoflunchy 13y ago

How does that compare to SSD failure rates? Are they much better?

KamiCrit 13y ago
I dunno if SSDs have been around long enough for reliability testing. Along with they have been advancing quite rapidly also so it is hard to stick with something and expect the info to be relevant down the road. All I know when it comes to SSD reliability, Intel is king.

protomyth 13y ago

In the last two year, I have stopped all buying of Seagate drives. We had a rather large rash of failures in RAIDs and desktops (50% of about 50 drives). Then the "every drive shipped by HP is failing" problems of the netbooks (70 total replace 50 drives) were also Seagate drives. We are basically an WD house now.

justin66 13y ago

> The drives that just don’t work in our environment are Western Digital Green 3TB drives and Seagate LP (low power) 2TB drives. Both of these drives start accumulating errors as soon as they are put into production. We think this is related to vibration. The drives do somewhat better in the new low-vibration Backblaze Storage Pod, but still not well enough. Another reason to avoid the WD Green 3TB: these drives aggressively put themselves to sleep to save power. It's literally a matter of streaming a video from disk and if the OS caches enough of the file, the drive will see there haven't been any accesses in a few seconds and stop spinning. The video will of course glitch when the cached data runs out and the drive needs to spin up. Great design.

B-Con 13y ago
Don't use the Green line for anything performance related, even if it's watching videos. The Green line is simply not intended for that. I think it's designed to be an occasionally-accessed backup drive, or something like that. The Black line is the performance-conscious line, look at the corresponding equivalents in that line. IIRC the Blacks are 10-15% more expensive, but they actually respond.
- IgorPartola 13y ago
  I recently bought two Green drives for my NAS not knowing the difference. Do your research :). Having said that, the NAS is working out pretty well so far with ZFS running on top of them.
  - frenchman_in_ny 13y ago
    I bought 3 Green 2.5TB drives for a Synology NAS that's lightly used -- and all 3 have had SMART failures in the ~18 months that I've owned them. WD just sent me a refurb'ed Green 3TB drive, let's see if it holds out better. I wish I had had access to this data before buying the Greens...
  - dpe82 13y ago
    I've had 4 Green 3TB drives in a raidZ for less than a year. First few months were great, then I had to replace two of them a couple months ago due to data errors and another just started failing so today (literally as I write this) I'm replacing the other two. Don't use Greens for anything that's on and could be active 24/7. The aggressive load cycling kills them.

undoware 13y ago

Aside from the incredible usefulness of the data herein -- thanks, backblaze! -- this is also the kind of marketing that I don't mind. Backblaze got egg on its face yesterday on HN when someone's critical report (rightly) got upvoted. Today they make up for it by giving us an interesting and useful data chart. I know it sounds weird, but for whatever reason I read this as demonstrating a high level of corporate responsibility and attunement to customers. They could have compensated by instead, say, dropping a few grand on buying journalists and 'reviewers', like Microsoft does. But they didn't. Instead, they were cool. To me that signals that they'll also take care of whatever problems they've had recently. (Note: I have no affiliation with Backblaze.) I'm currently not shopping for an online backup service but if that ever changes, I now have a good feeling about Backblaze, and I hope that other services take a similar approach to repairing customer relations when they are fraught. Shit happens, even to backup providers. It's how you respond that matters most.

lae 13y ago

If only I could find the Hitachi Deskstars for cheap.

cordite 13y ago

Interesting note on the WD 3TB Green. I have one in my rig, and whenever I do any disk access, I always have to wait about 5-8 seconds for it to spin up every other half hour. It seems to aggressively turn off. I have my base system on an SSD, and my games and other things on this 3TB drive. I guess such spin up times would be unacceptable in backblaze's environment.

rythie 13y ago

I wish Backblaze would provide some sort of Amazon S3 competitor, Amazon always seems very overpriced.

atYevP 13y ago
Hypothetically, what would you be looking for in an S3 competitor? Specifically: 1. How much data do you have? 2. What would you use it for? 3. How important is it if the API isn't the same as the S3 API? 4. Any specific certification requirements? 5. Would you have any SLA requirements? 6. Specific performance metrics? Think Amazon S3 vs. Glacier. 7. Are redundant data centers important? Hypothetically!
- rythie 13y ago
  I think the main problem is cost, Amazon's costs prevent startups like Everpix from existing as does it for video startups and various other startups that require a lot of storage. I think Digital Ocean's disruption of the very established VPS market is the model to follow. Personally I'd like to have all my data in the cloud. 1. 300GB growing at 25-50GB/year 2. Backup / cloud storage 3. Possibly, though I'd like SFTP access really 4. no 5. Don't lose my data, probably not down for more that 3-4 hours in one go. 6. Enough to stream video to back to my box 7. Not really, I'd assume the chance of data center being down is pretty small. Hitachi 500GB USB $2.4/month (based on $60 drive with 2 year warranty) Backblaze $5/month [Unlimited] Crashplan $5/month [Unlimited] Amazon charge $22.80/month Dropbox $50/month [500GB] (though I still need a local copy) Digital Ocean instance $320/month I think there is opportunity to come in at half of Amazon's price or less and that could lead to a new set of start-ups that could build on that.
  - atYevP 13y ago
    Thanks for taking the time to answer! Interesting use-case!

happycube 13y ago

Two things I never like to see together on a hard drive: "Seagate" and "Made in China"

ck2 13y ago

I'm a big fan of single platter drives, I just buy the biggest single platter at the time. Which is currently the WD 1gb blue. Very fast, very cool running.

krakensden 13y ago
1GB? that seems... awful small.
- scholia 13y ago
  Massive compared to the 10MB drive in the IBM PC/XT....
- baddox 13y ago
  And even 1TB, which is presumably what he meant, feels very small to me.
PavlovsCat 13y ago
Any particular advantages of having a single platter? Also, I assume you meant to say 1TB ^^
- grey413 13y ago
  The logic is that fewer moving parts means a lower failure rate. It amounts to paying a premium for longer-lived drives. How much you value that is a matter of taste and means.
paol 13y ago
The fewer platters the better for vibration and noise as well. Though I wouldn't necessary restrict myself to single-platter drives, 2 platters is generally fine.
theandrewbailey 13y ago
It's really unfortunate that WD has completely halted expanding capacity on their Blue line. Meanwhile, my WD6400AAKS (640GB blue, 2 platter) keeps marching on.

deleted 13y ago

[deleted]

goofygrin 13y ago

Seagate Barracuda Green (ST1500DL003) 1.5TB 51 0.8 120.0% I've got one sitting on my desk that seagate sent me as a RMA return. Guess it won't be going back into our RAID.

cbr 13y ago

Any stats on power consumption? Over 5 years the difference between a drive that uses 6 Watts and one that uses 7 is 44kWh or about $5. Double that to include cooling costs and saving a Watt should be worth something like $10 to you, so a more expensive more efficient drive could be worth it. Do these drives all use similar amounts of power?

ars 13y ago
Cooling costs are an extra one third (and even that is only during the summer), not even close to double.
- gibybo 13y ago
  Do you have a source or at least an explanation for that? I've always thought it took at least as much energy to cool something as it did to heat it (because AC is not 100% efficient), but I've been pretty wrong about thermodynamics in the past so I'd like to dive a bit deeper here. Maybe this is only true for AC, and not water cooling being pumped to passive radiators on the roof? EDIT: Here: https://www.google.com/about/datacenters/efficiency/internal/ https://www.google.com/about/datacenters/efficiency/internal..., Google is claiming a 'Power usage effectiveness' of 1.11 across their data centers. This implies a cooling cost <= 11% of their server costs which I found quite surprising.
  - TheCoelacanth 13y ago
    Hard drives are going to run hotter than the outside temperature even in the middle of the summer, so they are going to get quite a bit of natural cooling for free. The cooling they have to do is just a supplement to the cooling that happens naturally.
  - ars 13y ago
    It's pretty easy to calculate. Just find a window A/C and check the BTU it's rated for and the wattage. Convert BTU to watts, then do wattage to power the A/C divided by the watts of cooling power. Then note that this is the worse case scenario - normally it does better than that except on the very hottest days of the year.

halayli 13y ago

Lesson from this: buy yourself a hitachi.

caycep 13y ago

I'm just an amateur at statistics, but I would think this would be a useful set to do generalized linear mixed models on, to see what factors could be statistically significant (manufacturer, model, factory location, etc etc etc)

ChuckMcM 13y ago

Great post, as you get bigger populations of drives you can get a lot more visibility into their overall reliability. If there was one thing I could add to the analysis would be to split out the drives by serial number and split them out by firmware. Sometimes you find that all of the 'problem' in a set of problem drives is a single range of serial numbers. We've had similar experiences with replacement drives, they are, by and large, significantly less reliable than "new" drives. And last bit, we've got Western Digital drives here (a mix of 2.0 and 3.0 TB ones) They have been pretty solid performers for us.

bovermyer 13y ago

I am very impressed by the transparency here, and appreciative of the data. I'd never heard of Backblaze until now, but now I'll have to pay closer attention.

jpalomaki 13y ago

Posts like this are the reason why I trust Backblaze. In general I'm very suspicious about the unlimited offerings. When they tell the technical details I get the feeling they are up to task and not just reselling S3 and hoping people pay but don't actually use the storage.

jader201 13y ago

> We are focusing on 4TB drives for new pods. For these, our current favorite is the Seagate Desktop HDD.15 (ST4000DM000). We’ll have to keep an eye on them, though. Historically, Seagate drives have performed well at first, and then had higher failure rates later. I'm a little surprised that they actually did the analysis to determine the Seagates tend to fail more, yet they are still putting most (or at least, quite a bit) of their faith in those. Based on their own data, I would likely avoid those, or at least start leaning more toward Hitachi and WD. Or maybe the initial cost of those is so much better that it compensates for any long-term expense.

gatehouse 13y ago
You also have to consider that when the drives eventually fail, they will be replaced with hard drives of the future -- which will presumably be cheaper than the HDD of today. I.e. they depreciate quickly.

InclinedPlane 13y ago

Back in 2007/2008 or so I bought a pair of Seagate 7200 500gb drives and got bit hard by their extremely broken firmware. I haven't bought seagate drives since then, it's sad that their overall QC is still so terrible across the board.

madads 13y ago

Should consider what Jacob Applebaum revealed in his CCC preso about Hard drives: http://youtu.be/vILAlhwUgIU?t=46m25s http://youtu.be/vILAlhwUgIU?t=46m25s Cross out WD, Seagate, Maxtor and Samsung drives. Hitachi wins!

gnoway 13y ago

As an administrator with an admittedly smaller sample size, this lines up with my experience with the Seagate 1TB Barracuda ES.2, 15K.6/15K.7-era Cheetahs and some normal consumer Barracudas as well. All of the nearline and enterprise stuff had 5 year warranties, and over the past 5 years we've replaced over 25%. I've sworn off Seagate altogether, at least until they demonstrate a commitment to producing more reliable drives. I will not willingly buy them for the datacenter, and I won't buy them at any price for the home, RAID or no RAID.

dshep 13y ago

Dang! I just bought WD Green 2TB WD20EZRX drives (x4) for my NAS.

edward 13y ago

Price per TB: http://edwardbetts.com/price_per_tb/ http://edwardbetts.com/price_per_tb/

benrapscallion 13y ago

Along these lines, I have done a monetary analysis; http://perenniallycurious.com/centspergb.html http://perenniallycurious.com/centspergb.html

chemmail 13y ago

Shouldn't Toshiba be the drives to buy since they took over Hitachi's 3.5 drive factories when WD took over due to antitrust? http://www.wdc.com/en/company/pressroom/releases/?release=f8697da0-111a-49e8-a484-5266d18526ab http://www.wdc.com/en/company/pressroom/releases/?release=f8...

csense 13y ago

A better presentation of this data would show a failure rate for each brand and month/year of purchase. For extremely simple devices like resistors or incandescent light bulbs, failure rate is relatively constant over the lifetime of device -- the chance of a functioning resistor with 10 hours of use failing during the next hour is the same as the chance of a functioning resistor with 1000 hours of use. For complex devices with lots of interdependent parts, some of which are mechanical, the failure rate changes over time. There's an "infant mortality" or "lemon" phenomenon, where relatively new devices have higher defect rates (because fabrication and shipping sometimes result in imperfections which quickly cause failures), followed by a steep dropoff in failure rates (because observing a device operate correctly for dozens of hours is strong evidence that it doesn't suffer from a failure mode which often results in infant mortality). Then there may be an increase in failure rates later, especially with devices that are partially or wholly mechanical (wear or damage type problems which do not cause immediate failure, but make it easier for a failure to occur). You need empirical data to be quantitative about this curve, and it sounds like Backblaze has it, but their presentation in this article doesn't show it.

hga 13y ago
As I recall one of the studies of a few years ago, the one based on supercomputers, not Google's, showed there was very little infant mortality, and wear clearly set in after roughly one year in service. The results were quite striking, and nothing like the bathtub curve many expected and that you sort of sketch out.

abcd_f 13y ago

Interesting stats, but I wonder if the drive usage pattern at BackBlaze is somewhat different from that of a home user. In terms of how they seek, read and write. Not that they put more mileage on them faster, but that they might be doing something fundamentally different from how these drives are tested by their manufacturer. Lots of bulk in-sequence writes or something else. The 5-7 times difference in failure rates between leading manufacturers is frankly hard to believe.