4 ms·

Dispelling Backblaze's HDD Reliability Myth

20 points by Freaky 13y ago

lytfyre 13y ago
>"Backblaze claims that drive temperature doesn't affect drive life. That is counter to the observations of many others, including drive manufacturers." Google has a whitepaper[0] on a large deployed sample size of drives which disagree with the author's claims here. I agree that more data from backblaze would be nice, but their claims do seem to have more plausibility than the author suggests. (If he has methodology issues with the Google paper, that would be more interesting) [0]- http://static.googleusercontent.com/media/research.google.com/en//archive/disk_failures.pdf http://static.googleusercontent.com/media/research.google.co... (pdf warning)
- nwh 13y ago
  There's a typo in that link. http://static.googleusercontent.com/media/research.google.com/en//archive/disk_failures.pdf http://static.googleusercontent.com/media/research.google.co... I'm more inclined to believe Google's results than anybody elses, the amount of data they have is staggering and the results they got are fairly clear cut. I personally don't bother cooling spinning disks, it just seems like a complete waste of time and effort based on that paper.
0x0 13y ago
Warning: Browsing this site on ios mobilesafari will randomly taskswitch into the app store app, onto the buy page for some ios game.
- MBCook 13y ago
  That has been happening to me more and more recently. It's amazingly obnoxious.
techsupporter 13y ago
I'm curious why the author believes that beating the ever loving crap out of drives in harsh conditions and seeing which ones come out, overall, on top creates a myth. Perhaps the Backblaze article is better headlined "which hard drives seem to hold up well under pressure." Consumer environments usually have crappy, unreliable power (power strip chained to power strip plugged into an outlet from the 1960s), random vibrations (especially with kids), and poor air flow conditions (who here has done Family IT and had to carve the dust out of the case stuck behind a non-moving fan?). The only thing a consumer environment lacks from Backblaze's comparison is 24/7 read/write operations.
- mnordhoff 13y ago
  I would read an article about how bullet-resistant different models of car are, but I wouldn't completely change my purchasing decisions, as I avoid bullets. (Whoa, I didn't even mean to use a car analogy.) "Consumer environments usually have crappy, unreliable power ..." I would hope that Backblaze is on reliable, stable, data center power -- though knowing how cost-conscious they are, perhaps not -- so that's one aspect where they wouldn't be able to provide good information.
  - ricardobeat 13y ago
    You got it backwards. You live in Kabul, so the article on bullet resistance is completely relevant. The point is that Backblaze's deployment conditions are way more similar to a domestic user's than other enterprise deployments.
    - mnordhoff 13y ago
      More similar, yes, but maybe not similar enough. I wonder if it's the pathological vibration killing the Seagates. If so, you don't want to use Seagates in a file server sitting on top of your washing machine, but maybe they're fine for your living room media box. Edit: Unlike Backblaze, I have no hard numbers, and can only pull suppositions out of my ass.
- wmf 13y ago
mnordhoff 13y ago
"With varying numbers of drives for each model, it is possible that some bad batches may have made their way into the sample pool, thus further skewing the numbers." So, to measure drive reliability, the author recommends that you don't count unreliable drives, since it may "skew the numbers"?
deleted 13y ago
[deleted]
cuhaos 13y ago
This Tweaktown article offers nothing in the way of factual evidence to discount any of the hard work Backblaze put into that blog post. Shame on Tweaktown for poo poo'ing on Backblaze's effort.
pcurve 13y ago
I read the article with great anticipation, but the author fails to poke any more holes in Backblaze's test that they themselves haven't already caveated. Backblaze already acknowledged that their usage environment doesn't reflect that of normal consumer, yet the author harps on the point. The author tries to debunk Backblaze's claim that it found no correlation between drive temperature and failure rate. Yet he presents no evidence that Backblaze's methodolgy was wrong. Backblaze had large enough sample to have even distribution of drives in various temperature, rack, and load environment. I doubt they put all Seagate drives in the middle shelf of Ver. 1 racks where it was the hottest, but put all Hitachi drives in the bottom of the Ver. 3 racks where it was cooler and more stable. Unless there is another study that shows failure rate of these drives in normal consumer environment, Seagate's 3-5x failure rate should give people a pause.
- wiredfool 13y ago
  I find it particularly galling that "Everybody knows temperature is important" is the rebuttal to a bunch of data saying "we didn't see an effect". If you've got data, post it. If not, it starts to sound like astroturf. Fwiw, with my small sample size, I agree with backblaze. Seagates appear to be failing more, and avoid wdc greens.
  - zzzcpan 13y ago
    > and avoid wdc greens Why? Backblaze reports 3.6% AFR on wdc greens over 4.4 years, which is pretty good.
    - wiredfool 13y ago
      I think they had issues with early death from vibration, and they don't perform well in anything like demanding performance situations. Though that may just be my faulty memory of he The original piece.
    - sexmonad 13y ago
      Anecdotally, at my last job, we had used them on several of our internal file and VM servers (because they were cheap), and they had a nasty habit of falling off the bus overnight, causing the RAID controller to go berserk.
TYPE_FASTER 13y ago
That article was just like reading the Backblaze post, if the Backblaze post had been split into three pages and covered in ads.
nkurz 13y ago
This article is astonishing. I try hard to give the benefit of the doubt to authors (as opposed to salespeople, speechwriters, and advertisers), and assume that they have good intentions and believe what they are saying. But I can't find any explanation for this article other than paid hit-piece or personal vendetta. Here's the article's conclusion: The data from Backblaze should not influence a purchasing decision by any consumer, regardless of what type of drive they are purchasing. The innumerable variables, and lack of documentation, ensures the results are unreliable. Even for the winners, the results aren't good; the failure rates are exponentially higher than those observed in the real-world. One should question whether these companies could survive financially with the massive warranty return rates in real-world scenarios. And here is some of the logic: It is the release of the data, in handy charts and graphs that encourage misrepresentation, which brings out the data-storage stickler in me. HDD manufacturers spend billions of dollars in R&D, and their labs are designed to characterize and measure the reliability and endurance of their storage solutions. This last line converted from to "undecided" to "paid propaganda", as I doubt that vendettas come with such clearly presented "suggested talking points". It's worth reading if just to see such a stunning example of the art form.
- fatrachet 13y ago
  A third option other than hit-piece or vendetta would be just page views. A lot of tech writers and bloggers just write counter pieces to popular articles in the hope to get read, spread and shared by whoever dislikes or disagrees with the original article. It's become very common to see articles like "Why [insert yesterdays top HN post] is wrong/will never work/is flawed/..." on HN and reddit.
  - nkurz 13y ago
    I'm willing to believe that page views are his and his employer's intent, and I'm willing to believe it would be titled as it is, the level of vitriol in the content makes me think there must be something more.
- notacoward 13y ago
  Data-storage stickler, indeed. He has been in the storage industry for barely over a year, with no achievements of note. Before that he was a restaurant manager. He's not just making an appeal to authority, but an appeal to risible authority. Whatever the flaws in Backblaze's data might be, it's still data and that wins over completely empty blathering.
pg_is_a_butt 13y ago
uh.... "all hard drive articles are lies".... "here is my new hard drive article" ignorant hypocrites are ignorant hypocrites. #OFFTOBUYANOTHERWESTERNDIGITAL
josephlord 13y ago
Of course it isn't a large scale scientific experiment with controls in place for all the variables they are running a business and the data is a by-product (albeit one which they can use to optimise their own business). But it is the largest scale testing I'm aware of that publishes the manufacturer names. For that alone it is valuable. I also like the fact it is harsher than normal environment, I don't want the most reliable disk if it never gets touched or used or to have to baby it.
xlfe 13y ago
The author of this article (Paul) appears to ignore the law of large numbers - while Paul's arguments against using the figures from Backblaze as measures of drive reliability would hold if Backblaze's analysis was based on a small sample size, because their sample size is relatively large, Paul's criticisms don't hold. Paul's argument is that any analysis of hard-drive reliability should control for factors like vibration and temperature as these vary in all environments - but in fact, because Backblaze has taken large (enough) sample of drives, these variations are already controlled for (ie cancelled out). Sure the consumer is probably never going to experience a failure irrespective of whether the buy a seagate or WD drive - but that's not the point - the point is that the long tail of the distribution, WD and Fujitsu beat Seagate for reliability. That is still valuable information for people, even if they aren't going to be using their drives in the same type of environment as Backblaze.

20 points by Freaky 13y ago

>"Backblaze claims that drive temperature doesn't affect drive life. That is counter to the observations of many others, including drive manufacturers." Google has a whitepaper[0] on a large deployed sample size of drives which disagree with the author's claims here. I agree that more data from backblaze would be nice, but their claims do seem to have more plausibility than the author suggests. (If he has methodology issues with the Google paper, that would be more interesting) [0]- http://static.googleusercontent.com/media/research.google.com/en//archive/disk_failures.pdf http://static.googleusercontent.com/media/research.google.co... (pdf warning)

nwh 13y ago
There's a typo in that link. http://static.googleusercontent.com/media/research.google.com/en//archive/disk_failures.pdf http://static.googleusercontent.com/media/research.google.co... I'm more inclined to believe Google's results than anybody elses, the amount of data they have is staggering and the results they got are fairly clear cut. I personally don't bother cooling spinning disks, it just seems like a complete waste of time and effort based on that paper.

0x0 13y ago

Warning: Browsing this site on ios mobilesafari will randomly taskswitch into the app store app, onto the buy page for some ios game.

MBCook 13y ago
That has been happening to me more and more recently. It's amazingly obnoxious.

techsupporter 13y ago

I'm curious why the author believes that beating the ever loving crap out of drives in harsh conditions and seeing which ones come out, overall, on top creates a myth. Perhaps the Backblaze article is better headlined "which hard drives seem to hold up well under pressure." Consumer environments usually have crappy, unreliable power (power strip chained to power strip plugged into an outlet from the 1960s), random vibrations (especially with kids), and poor air flow conditions (who here has done Family IT and had to carve the dust out of the case stuck behind a non-moving fan?). The only thing a consumer environment lacks from Backblaze's comparison is 24/7 read/write operations.

mnordhoff 13y ago
I would read an article about how bullet-resistant different models of car are, but I wouldn't completely change my purchasing decisions, as I avoid bullets. (Whoa, I didn't even mean to use a car analogy.) "Consumer environments usually have crappy, unreliable power ..." I would hope that Backblaze is on reliable, stable, data center power -- though knowing how cost-conscious they are, perhaps not -- so that's one aspect where they wouldn't be able to provide good information.
- ricardobeat 13y ago
  You got it backwards. You live in Kabul, so the article on bullet resistance is completely relevant. The point is that Backblaze's deployment conditions are way more similar to a domestic user's than other enterprise deployments.
  - mnordhoff 13y ago
    More similar, yes, but maybe not similar enough. I wonder if it's the pathological vibration killing the Seagates. If so, you don't want to use Seagates in a file server sitting on top of your washing machine, but maybe they're fine for your living room media box. Edit: Unlike Backblaze, I have no hard numbers, and can only pull suppositions out of my ass.
wmf 13y ago

mnordhoff 13y ago

"With varying numbers of drives for each model, it is possible that some bad batches may have made their way into the sample pool, thus further skewing the numbers." So, to measure drive reliability, the author recommends that you don't count unreliable drives, since it may "skew the numbers"?

deleted 13y ago

[deleted]

cuhaos 13y ago

This Tweaktown article offers nothing in the way of factual evidence to discount any of the hard work Backblaze put into that blog post. Shame on Tweaktown for poo poo'ing on Backblaze's effort.

pcurve 13y ago

I read the article with great anticipation, but the author fails to poke any more holes in Backblaze's test that they themselves haven't already caveated. Backblaze already acknowledged that their usage environment doesn't reflect that of normal consumer, yet the author harps on the point. The author tries to debunk Backblaze's claim that it found no correlation between drive temperature and failure rate. Yet he presents no evidence that Backblaze's methodolgy was wrong. Backblaze had large enough sample to have even distribution of drives in various temperature, rack, and load environment. I doubt they put all Seagate drives in the middle shelf of Ver. 1 racks where it was the hottest, but put all Hitachi drives in the bottom of the Ver. 3 racks where it was cooler and more stable. Unless there is another study that shows failure rate of these drives in normal consumer environment, Seagate's 3-5x failure rate should give people a pause.

wiredfool 13y ago
I find it particularly galling that "Everybody knows temperature is important" is the rebuttal to a bunch of data saying "we didn't see an effect". If you've got data, post it. If not, it starts to sound like astroturf. Fwiw, with my small sample size, I agree with backblaze. Seagates appear to be failing more, and avoid wdc greens.
- zzzcpan 13y ago
  > and avoid wdc greens Why? Backblaze reports 3.6% AFR on wdc greens over 4.4 years, which is pretty good.
  - wiredfool 13y ago
    I think they had issues with early death from vibration, and they don't perform well in anything like demanding performance situations. Though that may just be my faulty memory of he The original piece.
  - sexmonad 13y ago
    Anecdotally, at my last job, we had used them on several of our internal file and VM servers (because they were cheap), and they had a nasty habit of falling off the bus overnight, causing the RAID controller to go berserk.

TYPE_FASTER 13y ago

That article was just like reading the Backblaze post, if the Backblaze post had been split into three pages and covered in ads.

nkurz 13y ago

This article is astonishing. I try hard to give the benefit of the doubt to authors (as opposed to salespeople, speechwriters, and advertisers), and assume that they have good intentions and believe what they are saying. But I can't find any explanation for this article other than paid hit-piece or personal vendetta. Here's the article's conclusion: The data from Backblaze should not influence a purchasing decision by any consumer, regardless of what type of drive they are purchasing. The innumerable variables, and lack of documentation, ensures the results are unreliable. Even for the winners, the results aren't good; the failure rates are exponentially higher than those observed in the real-world. One should question whether these companies could survive financially with the massive warranty return rates in real-world scenarios. And here is some of the logic: It is the release of the data, in handy charts and graphs that encourage misrepresentation, which brings out the data-storage stickler in me. HDD manufacturers spend billions of dollars in R&D, and their labs are designed to characterize and measure the reliability and endurance of their storage solutions. This last line converted from to "undecided" to "paid propaganda", as I doubt that vendettas come with such clearly presented "suggested talking points". It's worth reading if just to see such a stunning example of the art form.

fatrachet 13y ago
A third option other than hit-piece or vendetta would be just page views. A lot of tech writers and bloggers just write counter pieces to popular articles in the hope to get read, spread and shared by whoever dislikes or disagrees with the original article. It's become very common to see articles like "Why [insert yesterdays top HN post] is wrong/will never work/is flawed/..." on HN and reddit.
- nkurz 13y ago
  I'm willing to believe that page views are his and his employer's intent, and I'm willing to believe it would be titled as it is, the level of vitriol in the content makes me think there must be something more.
notacoward 13y ago
Data-storage stickler, indeed. He has been in the storage industry for barely over a year, with no achievements of note. Before that he was a restaurant manager. He's not just making an appeal to authority, but an appeal to risible authority. Whatever the flaws in Backblaze's data might be, it's still data and that wins over completely empty blathering.

pg_is_a_butt 13y ago

uh.... "all hard drive articles are lies".... "here is my new hard drive article" ignorant hypocrites are ignorant hypocrites. #OFFTOBUYANOTHERWESTERNDIGITAL

josephlord 13y ago

Of course it isn't a large scale scientific experiment with controls in place for all the variables they are running a business and the data is a by-product (albeit one which they can use to optimise their own business). But it is the largest scale testing I'm aware of that publishes the manufacturer names. For that alone it is valuable. I also like the fact it is harsher than normal environment, I don't want the most reliable disk if it never gets touched or used or to have to baby it.

xlfe 13y ago

The author of this article (Paul) appears to ignore the law of large numbers - while Paul's arguments against using the figures from Backblaze as measures of drive reliability would hold if Backblaze's analysis was based on a small sample size, because their sample size is relatively large, Paul's criticisms don't hold. Paul's argument is that any analysis of hard-drive reliability should control for factors like vibration and temperature as these vary in all environments - but in fact, because Backblaze has taken large (enough) sample of drives, these variations are already controlled for (ie cancelled out). Sure the consumer is probably never going to experience a failure irrespective of whether the buy a seagate or WD drive - but that's not the point - the point is that the long tail of the distribution, WD and Fujitsu beat Seagate for reliability. That is still valuable information for people, even if they aren't going to be using their drives in the same type of environment as Backblaze.