8 ms·
An Empirical Analysis of Hardware Failures on a Million Consumer PCs
- josephturnip 14y agoInteresting stuff. You can improve reliability by running your system at a lower speed. Here's a blog post with a summary of some of the conclusions of the paper above: http://grano.la/blog/2012/06/improve-the-reliability-of-your-pc/ http://grano.la/blog/2012/06/improve-the-reliability-of-your... (Disclaimer: that's my company's blog) One question I still have is whether the switching of CPU frequencies has any effect, or if it is only the average speed that correlates to the reliability. Anecdotal evidence suggests that this is the case, but it could be an area for further research.
- latch 14y agoIs there a compelling reason for this to be a PDF rather than HTML? I'm genuinely curious.
- mbafk 14y agoI guess they just uploaded the final version they published in EuroSys 2011. Like all ACM conferences I know about, PDF is the way it is done. They would have to reformat for HTML.
- josephturnip 14y agoMost research papers are disseminated as PDF in my experience. This was a EuroSys 2011 paper I believe.
- scott_s 14y agoAs mbafk and josephturnip state, they simply put online the same copy that was published in the conference. Academic conferences typically publish papers in PDF form. But, that doesn't actually answer your question, which I think is "Barely." I feel silly preparing PDFs for publication when I know that most people will read it on their computer, not print it out. Many conferences no longer even have an actual, physical copy of the proceedings, instead just giving out USB sticks with all of the PDFs. (Which is what we want anyway.) I think it would be fantastic if there was a standard HTML5 template that researchers could use to publish their papers. There are Latex-to-HTML compilers, but I've never been impressed with the results. I think people outside of academia would be more likely to read our papers if they were in HTML rather and PDF.
- mbafk 14y agoI agree it would be great if there was a good HTML template the IEEE or ACM used. Even using Latex you spend ages making it look exactly the way you want and don't want to upload a dodgy HTML version.
- scott_s 14y agoThe major difficulty, I think, are figures. How can we maintain keeping figures near the text that talks about them, and also allows a high density of them? The best I can come up with is a narrow column of text on the left (similar in size to a column of text in an ACM or IEEE double column format), and a larger column on the right with figures. The difficulty is in anchoring the figures in the right column with the text. But then you may have a bunch of figures piled up in one place, and a sensible layout becomes difficult to do.
- Dn_Ab 14y agoI am outside academia but I read a lot of papers. For most people outside I suspect the biggest problem is paywalls and not format. If you don't read a paper because it is a PDF and not HTML then you weren't really interested. With chrome, it is not even an annoyance as when you had to load adobe. One reason for PDFs is as you mentioned, latex to HTML results are typically poor. Diagrams are another difficulty that don't have an easy HTML solution. Other reasons I prefer PDFs are: though I never print, I often save papers to disk since I don't always find the paper when I go searching the second time (especially if it is months or even years after), there is a real benefit to being able to read a paper offline - I don't always have a connection to the net when I want to read and lastly, if you have an ereader such as the kindle, pdfs render well on them.
- scott_s 14y agoPaywalls are probably a bigger issue, but it's still friction. Anyway, modern browsers give the option to save all of the images along with the HTML file.
- andreasvc 14y agoMathematical formulas are a big reason; wikipedia uses images which look passable unless you start zooming in. Another reason is that with HTML you're never sure if you'll end up with the fonts as they were intended. Lastly, because HTML is not a page-oriented markup format, the presentation in general can differ from system to system; e.g., you can't say "on line 2 of page 3".
- scott_s 14y agoYou may not have seen MathJax: http://www.mathjax.org/ http://www.mathjax.org/ The main drawback is that I noticed a delay before it fills in the correct math notation. But the results are quite impressive.
- kristaps 14y agoInteresting, too bad the power supplies could not be controlled in their setup, as a wonky power supply can unleash all kinds of gremlins that look like failures in components down the line.
- slug 14y agoMost of my desktop/server computer failures or abnormal behavior are due to power supplies not working properly, even when protected by a decent UPS.
- Avitas 14y agoI have been telling my staff for probably 15 or so years that likeliest cause of PC failure is (in the following order): - Power supply - Hard Drive Ranking near these are sleeve bearing fans with ball bearing fans a close second--for CPU and case cooling. In rough order, I would say that the following is my estimation of other common component failure sources: - Removable Drives (floppy, optical, etc.) - Video Card (if separate) - Motherboard - RAM - CPU These are for our corporate PCs which have been Compaq, Dell, IBM, HP, Lenovo and a few other brands. For our brand name and whitebox server hardware, it's pretty much the same... if something is going to fail, it's going to be a power supply or a hard drive. In fact, I don't ever remember a single server motherboard, RAID controller, RAM stick, CPU or other component ever going bad in a server. I wonder why they would leave out statistics relating to power supplies when they are, in my experience, the component with the greatest failure rate.
- acqq 14y agoThere are a lot of insights in the paper, but I'd really like to know about this: "The table shows that CPUs from Vendor A are nearly 20x as likely to crash a machine during the 8 month observation period when they are overclocked, and CPUs from Vendor B are over 4x as likely" Obviously it's 5 times difference in probability to have unstable system if overclocked between Intel and AMD but they don't say which one is better. Anybody knows?
- reitzensteinm 14y agoAs a stab in the dark, I'd say that it's Intel's chips that perform more stably while overclocked, all else equal, because Intel comfortably holds the performance crown, and is thus very conservative in their binning. Right now, Intel's fastest desktop chip is an i7 990X, which is $1,029 on Newegg. AMD's is an FX-8150, at $199. Intel prices pretty fairly against AMD on the price/performance curve where AMD has a competitor, eg, the Core i5 3550 at $209 generally outperforms AMD's fastest chip. Pricing then soars off into the sky. Which is to say, if AMD bumps the speed of their CPUs, they'll release a faster product, compete better against Intel (until Intel reacts), and make more money. If Intel bumps its speeds, they're competing against nobody but themselves, so they usually don't bother. Therefore, you usually see Intel quite conservatively binning their chips, and they have a lot of headroom. It's not unusual to have an AMD chip that can't go 200mhz faster on air cooling, and to have an Intel chip that can go well in excess of 1ghz faster. So all else equal, bumping Intel chips is less likely to be an issue. Now, there are some other factors at play here. Firstly, hardcore overclockers pretty much only buy Intel chips. Also, people that are serious often turn up the speed until right before the moment at which the chip starts getting SuperPI errors (i.e. errors at nearly maximum load). But my totally uninformed gut feeling is that the majority of overclocks aren't done that seriously (if they are, it could actually reverse this analysis). If we assume that, turning the knob up on an Intel chip without sophistication is much less likely to end badly. The crashes in their paper are not very frequent on average (months between), which isn't necessarily bad enough to revert to old CPU speeds even if the user knows that's what is going on.
- zipdog 14y agoWhen Google did a massive analysis of hard-drive failures they also didn't publish manufacturer names, because they felt that it would tarnish the company name when it might just be a production run
- cs702 14y agoVery useful -- I will take this analysis into account when it's time to upgrade my current personal machine or configure the next one! Thank you for posting this here. The only thing I would have wanted to see but didn't in this analysis is how failure rates vary for different types of disk subsystem -- specifically, traditional hard drives versus the newer solid-state devices. I suspect, but don't know for sure, that the latter have much, much lower real-world failure rates in the first 30 days of total accumulated CPU time (TACT). The authors openly suggest that the sharp difference in failure rates between desktop and laptop machines may be due in part to their disk subsystems: "Laptops are between 25% and 60% less likely than desktop machines to crash from a hardware fault over the first 30 days of observed TACT. We hypothesize that the durability features built into laptops (such as motion-robust hard drives) make these machines more robust to failures in general." Alas, the authors don't delve any further into it. I'd like to see hard data comparing the real-world failure rates of both desktops and laptops using traditional versus solid-state disk subsystems.
- wazoox 14y agoSo far numbers I've seen seem to acknowledge a failure rate of SSDs in the same ballpark as spinning rust but it's only been slightly over a year that most SSDs are actually reliable. Many, many old models were absolutely terrible. Hence I think it may be difficult yet to draw reliable conclusions.
- wazoox 14y agoAmong other interesting insights: * a machine that crashed once is 100 times more likely to crash again; the more it crashes, the more it's prone to fail again. * overclocking significantly reduces reliability. One CPU vendor (AMD or Intel, but unspecified) is much worse in this regard, too. * conversely, underclocking improves reliability. * branded computers are more reliable than beige boxes. * laptops are more reliable than desktops.
- ippisl 14y ago>> branded computers are more reliable than beige boxes. This is meaningless. The real question is how those compare to beige box that uses a decent parts. But Microsoft definitely has interest in helping manufacturers of brand computers because piracy is more prevalent in beige boxes.
- scott_s 14y agoThis paper was written by researchers in Microsoft Research, and accepted in an academic conference. It is not marketing. To be clear, you are suggesting that the researchers were dishonest in order to help their company. I find this unlikely. Disclaimer: I am a researcher in a corporate lab.
- ippisl 14y agoI doesn't have to be dishonesty. It might just be bias(conscious or subconscious) not to waste time on this question. Even doctors show this kind of biases when advising people on choosing treatments(which is a much bigger moral issue).
- scott_s 14y agoTheir conclusions on "white boxes" are based on relatively straightforward statistical analysis of their data. In order for there to be bias against white boxes, one of the following has to be true: 1. Their data collection methods are biased against white boxes. Given the large sample size and the method of retrieving samples - automatically generated crash reports from users - I find this unlikely. They cover this point in section 3.2.1. 2. Their statistical analysis is flawed. I see no issues with it, nor did the reviewers. (Otherwise it wouldn't have been accepted.) 3. They lied. I am most skeptical on this one. It's disingenuous to gesture at researchers and allege bias based on their employer without actually saying how they are biased. Doing so is not valid skepticism, but prejudice.
- hollerith 14y agoToo bad CPU temperature was not part of the collection of data used in the study.
- hollerith 14y agoThe result most surprising to me is that laptops are between 25% and 60% less likely than desktop machines to crash from a hardware fault during the first 30 days worth of measurements. The much larger weight and volume of desktops would seem to make them easier to cool.
- josephturnip 14y agoThat caught my eye too.. perhaps it's because laptops are designed with being carried around in mind? Or, to draw on the other conclusions from the paper, because laptops generally have lower-frequency chips (and for that matter, slower memory and disks too)?
- Zenst 14y agoLaptops also have in effect a cleaner power source with UPS built in so any power glitch's get filtered on a laptop as standard compared to a desktop. Least I have found desktop's with a good quality PSU and UPS noticable more reliable than those without.
- jseliger 14y agoLaptops also have in effect a cleaner power source with UPS built in so any power glitch's get filtered on a laptop as standard compared to a desktop. I've heard this theory before but never seen it substantiated. Do you have a link to any research or articles that explains how this works?
- kyberias 14y agoHow it works? Laptops have a battery. Hence they are less likely to power off suddenly when mains voltage drops. And powering off suddenly increases the probability of some electronics in the system failing. No need to research, just simple electronics and logic.
- lostlogin 14y agoPull your laptops power cord out the wall, and plug it back in. Laptop unchanged. Dirty power supply, but laptop unaffected.
- Zenst 14y agoInteresting read though why can't Microsoft just tell me that my CPU or HD or memory is borking and suggest I RMA it instead of saying everytime - have you applied the latest updates, which I get to click unhelpful. Most important thing in a PC I have found for reliability above everything else is a good PSU, realy does make a difference on the hardware side as you give your kit cleaner power. Add UPS/surge protector and you can double the lifetime of kit. Least from experience I've had it has been noticable.
- Hoff 14y agoThe copy at Microsoft Research is offline. Here's another copy of the paper: http://eurosys2011.cs.uni-salzburg.at/pdf/eurosys2011-nightingale.pdf http://eurosys2011.cs.uni-salzburg.at/pdf/eurosys2011-nighti...
- rdmirza 14y agoMirror. (For people who tried to ctrl-f and found nothing, like me.)
- ChrisNorstrom 14y agoI'm having a hard time coming to terms with "Laptops less likely to crash from hardware fault than desktops" Everything we've learned from experience, surveys, and PC World magazines has showed the opposite. Heat kills hardware and laptops have their hardware packed together so closely that it generates lots of heat. Back then I remember reading something like 1 in 4 laptops fail in the first 3 years. Which was very believable, at the time I was in collage for game design & development. All 80 guys in our class had laptops from HP (with get this... Pentium 4s in them). Those laptops had a LOT of problems. They were basically portable heaters. So I guess laptops now have either much better cooling, much cooler CPUs or a combination. OR PCs are just terribly cooled.
- TazeTSchnitzel 14y agoJust a theory, but perhaps laptops have a shorter lifespan in general?
- dkarl 14y agoThat's an interesting point. Laptops get replaced faster, and when a laptop is replaced, it is typically left in a non-running state in which it will not experience or report errors. Desktops are often left running for months or years after they've been replaced, either through laziness or because of a desire to keep them around for a special purpose.
- mrb 14y agoNo. They only took into account the machines' first 30 days of service life: "we only count failures within the first 30 days of TACT (total accumulated computing time), for machines with at least 30 days of TACT."
- mertd 14y agoHere is the authors' hypothesis: We hypothesize that the durability features built into laptops (such as motion-robust hard drives) make these machines more robust to failures in general. Perhaps the physical environment of a home or of- fice is not much gentler to a computer as the difference in engineering warrants.
- mrb 14y agoWhen Microsoft, Google, or some university publish analysis of hardware failures across large numbers of machines, they always anonymize hardware vendors ("vendor A", "vendor B"). I understand the reasons (not alienating your hardware vendors), but will there ever be a research group who will disclose vendor names? Heck, I would pay for this information.
- rrreese 14y agoThis is dated, only applies to laptops, but it breaks down failures by manufacturer: http://www.electronicsweekly.com/blogs/engineering-design-problems/2010/02/one-in-three-laptops-fails-in.html http://www.electronicsweekly.com/blogs/engineering-design-pr...
- wmf 14y agoIMO such information would do more harm than good. By the time they could gather statistics, that model would be obsolete so the stats wouldn't help you buy new equipment. But the kind of people who are still griping about the Deathstar would use the information to troll non-stop.
- boxein 14y agoBUT if we had consistent poor performance for some vendor in a certain category, we could infer that their offerings for that category will also be poor in the future.
- erhh 14y agohttps://www.google.com/search?q=An+Empirical+Study+of+Redundant+Array+of+Independent+Solid-State+Drives https://www.google.com/search?q=An+Empirical+Study+of+Redund...
- stcredzero 14y agoI've always said that smart hardware tinkerers underclock. It produces less heat, and results in a quieter machine. I always suspected it improves reliability.
- wmf 14y agoOr you could save money and just buy a lower bin.
- stcredzero 14y agoWell, because heat dissipation is proportional to the square of the voltage, you end up giving up a little bit of performance but save a whole lot of heat. In experiential terms, you never miss performance but often notice a whole lot less fan noise. Buying from a lower bin, you're getting a crappier processor, which might give you less latitude to save heat. This would probably be worth measuring and writing an article about. Also, I tend to buy lower clocked processors as it is.
- wmf 14y agoI think it's likely that all lower bins are artificial, so e.g. underclocking a 2.4 GHz down to 2.0 is probably exactly the same as if you bought the 2.0. But yeah, it would be worth measuring.
- stcredzero 14y agoAh, I see. I wrote underclock. It's really undervolting that gets you the big win thermally. Underclocking should be done just as a means of achieving a greater undervolt. I just have these two things in the same mental bin.
- wmf 14y agoWhen you underclock properly (with SpeedStep) it also lowers the voltage... probably to the same voltage that the lower-bin processor would use.