53 ms·
This is consistent with prior work showing that SMART has no predictive value. Failure Trends in a Large Disk Drive Population, FAST '07 https://static.google
by jeffbee 6y ago
This is consistent with prior work showing that SMART has no predictive value.
Failure Trends in a Large Disk Drive Population, FAST '07
https://static.googleusercontent.com/media/research.google.com/en//archive/disk_failures.pdf https://static.googleusercontent.com/media/research.google.c...
- toast0 6y agoDidn't read your linked study, the linked blog post threw out the actual data though. Not surprising you get nothing of use when you start with: > Each attribute also has a feature named raw_value, but this is discarded due to inconsistent reporting standards between drive manufacturers. Sure, raw_value is inconsistent between manufacturers and sometimes models, but it's the most real data. Edit to add: reread the Google survey, which says more or less bad sector counts were indicative of failure in about half the cases; in 2007. One thing to note is drives made now may perform differently than those in the 2007 survey. Different firmware, different recording, better soldering (200x is full of poor soldering because of RoHS mandating lead free solder in most applications, and lack of experience with lead free solder leading to failing connections). I found, in 2013ish, with a couple thousand WD enterprise branded drives that just looking at the sum of all the sector counts and thresholding that was enough to catch drives that would soon fail. If your fleet is large enough, and you have things setup so failed disks is not a disaster (which you should!), it's pretty easy to run the collection and figure out your thresholds. IIRC, we would replace at 100+ damaged sectors, or if growth was rapid. Some drives in service before collecting smart data did seem to be working fine at around 1000 damaged sectors though, so you could make a case for a higher threshold. SSDs on the otherhand seemed to fail spectacularly (no longer visible from the OS), with no prefailure symptoms that I could see. Thankfully the failure rates were much lower than spinning disks, but a lot more of a pain.
- heipei 6y agoHmm, one could encode manufacturer and maybe even model as a categorical or one-hot encoded variable. Then again, the small number of failed drives in their dataset would probably make this hard.
- WrtCdEvrydy 6y agoI might disagree with this, I distinctly remember hearing that some SMART stats only update when you perform a SMART extended test. If the stats that matter only update after a test, and noone runs those tests, then is SMART the failure or is the lack of caring the failure.
- rasz 6y agoDo we need actual prediction? Ill settle for Just in time warning. This is what SMART parameter C4 gives you. You want to stop using the drive at the first sign of trouble, thats what C4 is good for. >Only one drive manufacturer had enough failed drives in the dataset to produce a large enough subset, which we’ll refer to as Manufacturer A Seagate, we will refer to this manufacturer as SEAGATE.
- jeffbee 6y agoI agree that the article fails to make an argument for why this prediction would be useful. What would you do with it, even if it was perfect?
- rasz 6y agoIf prediction would be possible you could do unethical things, like selling off units bound to fail in near future without disclosing this information. I wonder if there are any existing examples of such practices. Supermarket chains sell products near their expiration date at steep discounts, of course the difference is you know what you are buying and can still eat a 20 days until expiry 50% price Nutella jar.
- toast0 6y agoWith perfect (or good enough) prediction, you can replace drives before they fail, in controlled conditions in a maintenance window.
- jeffbee 6y agoAssuming your prediction is so good that it completely eliminates surprise failures, OK. But if your software can handle surprise failures then it can handle them, and in that case I don't see the value of the prediction. The only valuable thing I can see is if a system could predict that you were going to have widespread, coordinated failure of so many drives that your ability to reconstruct a stripe was at elevated risk. But such a thing is science fiction (as the article demonstrates) and you can approximate it quite well by assuming that all drives of the same make and model with the same in-service date will fail at the same moment.
- suifbwish 6y agoAs with many tasks assigned by stubborn executives, some things exist only as a monument to their own failure. Suuuuuuuuure boss we will build SMART exactly how you wanted..... to show you why it is a dumb idea