3 ms·
Since annual failure rate is a function mostly of age, it would be interesting to see a line chart of cumulative failure rate vs age. But since new drives are c
by zaroth 12y ago
Since annual failure rate is a function mostly of age, it would be interesting to see a line chart of cumulative failure rate vs age. But since new drives are continually being added to the population, there would be fewer drives in the data set as you moved up each curve.
I guess you could calculate confidence intervals at quarterly intervals, and so the error bars would get larger as age increases and 'n' decreases.
How would you calculate the CI for failure rate? It's not binomial or poisson, since failure rate goes to 1 over time...
A little searching turns up http://rmod.ee.duke.edu/statistics.htm http://rmod.ee.duke.edu/statistics.htm which I'm sure completely explains how to do this... (rolls eyes). I hate that this is how statistics is commonly taught. Knowing which distribution to use and applying it correctly can actually be intuitive if taught properly. It doesn't always need to be an exercise in alphabet soup / deriving from base principles.
- dugmartin 12y agoIt's been 20 years since I took a reliability engineering class but I believe the go to curve is the negative exponential. Here is a link from a quick Googling: http://www.quanterion.com/FAQ/Exponential_Dist.htm http://www.quanterion.com/FAQ/Exponential_Dist.htm
- darkmighty 12y agoThe exponential lifetime distribution is the model for ideal memoryless failures: the probability of failure in a dt interval is independent of current lifetime and have value lambda*dt. Those are as "random" failures as they get. I suppose hard drives are better modelled by a variable failure rate lamba(t), which should have a peak for the first few hours/days, settle down and then start growing quickly after a few months.
- jrapdx3 12y agoI'm not a statistician, but what you are describing is a situation in which the survival of the "subjects" (SSDs) is the crucial variable. How quickly the population declines is the question. This is similar to studies in medical research, e.g., how long do subjects live after an experimental cancer treatment. What I've seen used is the Kaplan-Meier non-parametric method (Kaplan-Meier plot) for the purpose. Maybe I'm just used to seeing these plots, but I find the method conveys information very effectively. More info can be found here: https://en.wikipedia.org/wiki/Kaplan%E2%80%93Meier_estimator https://en.wikipedia.org/wiki/Kaplan%E2%80%93Meier_estimator I agree that presenting the SSD life data in that way would be a good idea.
- cjslep 12y agoCouldn't a Weibull Distribution be used to determine the shape parameter (how the failure rate changes with time)? EDIT: More fine grained data is probably needed, granted.
- hyperbovine 12y agoUndergrad version: the lifespan of a single drive should be exponentially distributed with some rate lambda. The time to first failure from among k drives would then have an exponential distribution with rate k * lambda. The confidence interval for lambda is: http://en.wikipedia.org/wiki/Exponential_distribution#Confidence_intervals http://en.wikipedia.org/wiki/Exponential_distribution#Confid... Grad-level version: drive lifetime is exponential with an inhomogeneous (increasing, presumably) rate function lambda(t). Inferring lambda(t) is difficult without additional assumptions on the functional form. But potentially do-able. Real-life version: None of the classical distribution fit that well. (https://www.usenix.org/legacy/event/fast07/tech/schroeder/schroeder_html/ https://www.usenix.org/legacy/event/fast07/tech/schroeder/sc...) (This is survival analysis: http://en.wikipedia.org/wiki/Survival_analysis http://en.wikipedia.org/wiki/Survival_analysis)
- wcbeard10 12y agoHere's a blog post giving a tutorial on survival analysis using a relatively recent (and IMO very compelling) python library. Could be what you're looking for. http://camdp.com/blogs/lifelines-survival-analysis-python http://camdp.com/blogs/lifelines-survival-analysis-python
- kalleboo 12y ago> Since annual failure rate is a function mostly of age, it would be interesting to see a line chart of cumulative failure rate vs age They showed some of this data in an earlier post https://www.backblaze.com/blog/how-long-do-disk-drives-last/ https://www.backblaze.com/blog/how-long-do-disk-drives-last/ The "Drives have 3 distinct failure rates" graph is the most interesting, as it shows the result of the expected "bathtub curve" on the cumulative failure rate.
- hammock 12y agoYou can see cumulative failure rate vs age here (last graph) https://www.backblaze.com/blog/what-hard-drive-should-i-buy/ https://www.backblaze.com/blog/what-hard-drive-should-i-buy/
- rbcgerard 12y agoThe financial analogy is to think of failure rates in drives as equivalent to defaults on a pool of securities (say mortgages). obviously there are a number of ways to do this but the most simple is that since origination standards for loans vary through time (think build quality and design changes in the drive context) it seems to make sense to think about this from a vintage standpoint. i.e. are seagate drives installed in Dec'12 failing at the same rate as ones installed in March '13 after six months...