10 ms·

Analysis of SSD Reliability during power outages

168 points by yapcguy 13y ago

JohnTHaller 13y ago
The author of this study submitted it to Slashdot with the following summary: "After the reports on SSD reliability and after experiencing a costly 50% failure rate on over 200 remote-deployed OCZ Vertex SSDs, a degree of paranoia set in where I work. I was asked to carry out SSD analysis with some very specific criteria: budget below £100, size greater than 16Gbytes and Power-loss protection mandatory. This was almost an impossible task: after months of searching the shortlist was very short indeed. There was only one drive that survived the torturing: the Intel S3500. After more than 6,500 power-cycles over several days of heavy sustained random writes, not a single byte of data was lost. Crucial M4: failed. Toshiba THNSNH060GCS: failed. Innodisk 3MP SATA Slim: failed. OCZ: failed hard. Only the end-of-lifed Intel 320 and its newer replacement, the S3500, survived unscathed. The conclusion: if you care about data even when power could be unreliable, only buy Intel SSDs." Source: http://hardware.slashdot.org/story/13/12/27/208249/power-loss-protected-ssds-tested-only-intel-s3500-passes http://hardware.slashdot.org/story/13/12/27/208249/power-los...
- colechristensen 13y ago
  Did he do any control studies with spinning disks? I have my doubts that many of them would survive thousands of power cycles per day.
  - AmVess 13y ago
    There's probably not much of anything that would survive that many power cycles in a day. The test is pretty pointless.
    - ak217 13y ago
      It's not pointless, just misrepresented. SSDs enable new classes of applications for high-performance, non-power-protected computers. With an SSD and Intel ULV CPU, I can put together a cheap high-performance computer with no moving parts and no batteries. That has the potential to be very robust and have an expected lifetime of decades, but only if the SSD is rock solid.
      - cwzwarich 13y ago
        Which SSD is going to last for decades in a strenuous environment?
        ak217 13y ago
        Intel, apparently
        RyanZAG 13y ago
        This test really doesn't prove that at all. There are vastly different factors between power cycling in a single day and long term continuous use. You can't take results from one and get any meaningful information about the other.
        ak217 13y ago
        You're right. But it does show one way in which it's resilient. Given this information and what else I know about Intel SSDs, I would expect them to be the most resilient overall.
    - rlpb 13y ago
      What if I have thousands of disks deployed, and I suffer a single power loss? What are the chances that at least one disk will contain corrupt data after such an event? Instead of buying thousands of disks to test this scenario, one reasonable shortcut might be to repeatedly test a single disk. Even a spinning disk can handle a few thousand power cycles in its lifetime, surely?
      - AmVess 13y ago
        On power up, there's a pretty big spike in power usage as the controller gets its shit together. Normally, this is not an issue, since the SSD housing simply absorbs the heat and then goes about its business. The same holds true for spindle drives. What this guy did was test the drives in a manner they weren't designed for. Sure, I can drop Corvettes off a 20 story building, then bitch about the results, but that wouldn't change the fact that my test was flawed from the onset. All he did was subject something to an environment is wasn't designed for. Sure, I can drop a Corvette from a 20 story building, but there's nothing to be gained when the crumple zone is packed into the tail lights.
        zAy0LfpBZLC8mAC 13y ago
        Hu? Where should that spike in power usage come from?
        rational_indian 13y ago
        It comes from the decoupling/filter capacitors used in the DC power circuit. When the power is turned on after a sufficiently long time these capacitors are all uncharged and appear as "shorts" to the power source thus drawing large amounts of current. This initial surge current drops off as the capacitors get charged.
        zAy0LfpBZLC8mAC 13y ago
        Filter capacitors producing excessive heat due to inrush current? That would be bad filter capacitors indeed ;-) The claim was that an SSD somehow converted more electric energy into heat immediately after power-up which would damage the SSD, so real consumption, not just a current peak that goes into storage for later consumption. Normal-ESR electrolytics might have a heat problem when used at a few kHz in switching applications, but certainly not at 0.1 Hz.
    - ytjohn 13y ago
      From what they said about these reading sensor data, I'm picturing a factory, industrial, or field type scenario. I'm picturing a very small embedded computer with some sensors attached and an internal ssd. Referencing a 1.5gb OS like they did makes me think they just have a normal OS installed, but it's entirely possible that they have a modified kernel and fast boot process. Essentially, it sounds like they have a single unit that gets turned on, gathers sensor readings, and then get turned off. I'm guessing these are not handheld devices, otherwise they most likely would have a battery attached. If a non 24/7 factory, these could be turned off with the rest of the assembly line by someone throwing breakers off on the way out (very common).
  - nexox 13y ago
    I've seen a few different models of 2.5" Enterprise SAS drives go through a similar test (as a control,) and they were perfectly fine with no corruption after around 2 weeks of reboots at approximately 10 minute intervals. It's the SSDs that I worry about - some store their own firmware on the same NAND as the User's data, and allow it to become corrupted during power-loss events. Several cheap models won't last more than a couple days before they simply drop off the SATA bus and never come back.
Mithaldu 13y ago
Terrible title, terrible science, terrible conclusion. Spoiler: He only tested these five drives, only Intel survived, so if they are your candidates, apply his conclusion: Crucial M4 Toshiba THNSNH060GCS 60gb Innodisk 3MP Sata Slim OCZ Vertex 32gb Intel 320 and S3500 Notably missing is the Intel runner-up Samsung and probably others i'm not aware of, as well as other models.
- randall 13y ago
  A little harsh, don't you think? I'd say it's still useful to have 5 data points than 0.
  - Mithaldu 13y ago
    I'd have been gentle if his conclusion had included a measure of humbleness and at least kind of approached being somewhat accurate. :)
    - samplonius 13y ago
      And where was the "terrible science"? Requiring "humbleness" in posts seems to only apply to other people's posts, apparently.
      - emn13 13y ago
        The conclusion "buy intel" makes no sense. He tested far too few drives to make that conclusion, and many potentially safe alternatives went untested. Not to mention the fact that not all intel drives apparently are safe. So yeah; it's unfortunately bogus science - he's drawing invalid conclusions from a far too small sample.
      - vacri 13y ago
        Terrible science = small, restricted subject population, from which an ironclad blanket conclusion is made, that covers the entire population. One of the restrictions is price, which makes little sense in concluding so strongly for a measure involving quality. Not to mention that only a single specimen of a single model of each brand was tested. The top-voted comment in the thread points out some Intel models that don't have supercapacitors in them. Already the article is updated with a couple of other drives to test. I guess that wasn't "End of discussion" after all...
maerF0x0 13y ago
As many have pointed out: There are many applications where power cycling is extremely rare and also not a big deal. 1. Laptops have a built in UPS incase they're unplugged 2. Servers should have UPS incase they're unplugged or small outtages 3. Desktop drives shouldnt be trusted as the only copy. Though I imagine the data corruption would propagate to backups?
- MichaelGG 13y ago
  Unless the server UPS is inside the chassis, disk corruption on power loss seems like a very high risk. Datacenter-wide UPSes can fail (at one of XO's DCs, we had power loss 3 times in 2 months). People can accidentally unplug stuff.
- JohnTHaller 13y ago
  Servers can go down more than you think. Rackspace had an entire wing of their Texas data center go down hard 3 times within a couple months a few years back. I know because PortableApps.com's dedicated box was in that wing. It was an issue with the equipment that switches form external power to battery to generator power. And there was no secondary backup to that, so it would go down and take the whole wing all at once. This would corrupt our Drupal database and it would take about 8 hours to rebuild. So, having a reliable SSD with power loss prevention could be a lot more relevant than you expect.
- mike_esspe 13y ago
  Laptops are vulnerable to this problem, if you don't know about it. On my notebook I've lost a partition with Crucial M4. OS hanged, I did a hard reset and after reboot discovered data loss.
  - userbinator 13y ago
    I got an Intel SSD (G1 80GB) in my laptop to do OS development (drivers), which incidentally is a use case that requires lots of hard resets. Never had any problems with it, and it's been through dozens if not hundreds of hard reset cycles. In fact I got the SSD specifically because I didn't want to be spinning up/down a disk that much, and laptops tend not to have reset buttons (they really need one, IMHO.)
_mikz 13y ago
Would be nice to see this test for Samsung drivers which should be closer to Intel's quality.
CamperBob2 13y ago
It's long been conventional wisdom that you'd have to be crazy to buy anything but Intel when it comes to SSDs. This study isn't too surprising, in that regard.
- joenathan 13y ago
  I've never had any issues with Samsung or Kingston SSDs. The 840 EVO series and Pro series are the best performance for the money you can get.
  - yapcguy 13y ago
    Stick to the Pro. The EVO are based on TLC which are cheaper to manufacture and have a reduced life-span.
  - userbinator 13y ago
    Kingston used to rebrand Intel SSDs (same controller, just a smaller amouht of flash) for the "value market", so not surprising they were good.
  - rythie 13y ago
    The Samsung drives are poor when used on a NFS server with sync enabled - they are about 4x slower than the Intel 320 drives doing that. When used without sync, the same if no faster than the Intel drives - with the drawback of course that you've probably got corrupt data if you crash or lose power.
- kevinchen 13y ago
  I think it's more "buy only Intel if you have crazy reliability requirements and no periodic backups." For a typical user who just wants their stuff to load faster, it doesn't really matter: laptops shut down gracefully when they run out of power and power outages don't happen that often.
  - robocat 13y ago
    We only buy Intel for staff because: their time is worth a lot, any interruption due to a flaky drive can be expensive (especially if traveling), and to reduce worry. The time for me to research whether another Brand is reliable costs more than just choosing Intel. A no brainer decision. I.e. the savings on a cheaper drive are not worth the risks. Backup, always, of course.
deleted 13y ago
[deleted]
herf 13y ago
The Crucial M4 had no power-loss protection. The newer model (M500) does and would be a more interesting test.
slyall 13y ago
I really hate when people are sloppy with notation. 20GB = Twenty Gigabytes NOT 20gb = Twenty gram bits and: 20MB/s = Twenty Megabytes per second NOT 20mbytes/sec = Twenty milli-bytes per second ( If you got you B's and b's correct about you would need to write out "bytes" )
- jpk 13y ago
  My favorite is "5KB". The prefix for kilo- is a lowercase 'k'. Uppercase 'K' is Kelvin. I don't even know how to conceptualize a Kelvin-Byte.
  - wyager 13y ago
    >I don't even know how to conceptualize a Kelvin-Byte. I can actually see where this unit would be useful; in determining the probability of bit rot. The more kelvins you have, and the more bytes you have, the more likely you are to flip a bit due to random fluctuations. temperature*storage capacity = Kelvin-Bytes.
  - zAy0LfpBZLC8mAC 13y ago
    Once you have that figured out, try to conceptualize Kelvinbyteohms aka Kelvinbytes per Siemens!
- illicium 13y ago
  If you want to be really pedantic, differentiate SI giga/megabytes as GB/MB and "binary" gibi/mebibytes as GiB/MiB
  - StavrosK 13y ago
    So 1024 bytes is 1 kiB?
    - jpk 13y ago
      Yep, see: http://en.wikipedia.org/wiki/Binary_prefix http://en.wikipedia.org/wiki/Binary_prefix
mandeepj 13y ago
My brand new crucial M4 ssd stopped working after few days. To be precise, the system would show the drive for about 2 minutes after each reboot then after that the drive would not show up anywhere at all. Whether you try in windows explorer or device manager or any where else. I contacted their support team but they never replied.
- sitkack 13y ago
  Crucial reportedly has better support than that. My M4 would crash after 21 minutes (exactly-ish) over a year continuous service (100+GB/day), the latest firmware fixed the problem and increased speeds. The USB firmware updater was a god send compared to the vile vile hoops I had to jump through to upgrade my OCZ drive. I currently have an Samsung 840 1TB and has been rock solid, it replaced a 250GB intel 320. Afaik Samsung is the only manufacturer that owns the whole supply chain, flash + controller.
  - kalleboo 13y ago
    Toshiba also do both NAND and controllers.
- codecop 13y ago
  I got my Crucial M4 4 month ago and it stopped working too. BIOS dont show it. Got replaced with new one. After 3-4weeks failed again. Bios shows SSD but testing shows many errors. I will send it again or will try update firware.
strlen 13y ago
What I find rather annoying is that manufacturers/marketers rarely state whether or not an SSD has a supercapacitor or not. Here have been my finding: - OCZ Vertex 4 SSD does (but this is not advertised) - OCZ Vertex 3 does not, Vertex 3 Pro does - Ditto for Vertex 2 / Vertex 2 Pro - OCZ Deneva(2) R does, other OCZ Deneva/Deneva 2 do not -- and in the case of Deneva 2 R this is advertized - Intel 320 does (but is only sata II), but this is not advertised at all - Intel 520/530 does not - Intel 330/335: unclear - Intel S3700/S3500: does and this is advertised Background: I was looking for an SSD to use for a ZFS intent log ("ZIL" -- ZFS's write ahead log) -- my requirements were a sandforce controller (or equivalent) and toggle NAND (so that I could use the same disk for both ZFS cache and the ZIL), and a supercap. This was surprisingly hard to find. What I'd like to see is: 1) Data on which SSDs have supercap 2) Data on which SSDs actually honour cache flush request (then a UPS + forcing a cache flush request upon power failure + redundant power supplies/multiple replicas in a distributed system) would suffice. 3) Best yet: have an API for for check whether or not an SSD has a supercap, whether or not the supercap still holds charge, and policy for honouring cache flush requests. Let the OS decide based on a policy I set. If you are a consumer, the practical recommendation is not very different from the practical recommendation I'd give to anyone using spinning disks: RAID1/1+0/RAIDZ(-2) (with SSDs coming from different patches so that they do not wear out at the same time), UPS (for power outages), backups (against yourself and against power supply failures). For production: obviously use a UPS, put the WAL of your database on an SSD with a supercap, make sure that your database fsync()'s the WAL at a reasonable interval (on every transaction is probably unreasonable, but so is once an hour), and use a distributed system that replicates the WAL. If using a distributed system with (semi-)synchronous WAL replication is not an option and losing incremental data is not acceptable, use redundant power supplies.
- AnthonyMouse 13y ago
  > If using a distributed system with (semi-)synchronous WAL replication is not an option and losing incremental data is not acceptable, use redundant power supplies. Also be sure to plug each redundant power supply into a separate UPS, and avoid equipment that has redundant power supplies on a single feed. UPS failures are a bear.
rlpb 13y ago
Why must SSDs have "power loss protection" in the form of a battery or supercapacitors to finish writes? Can they not simply cache writes but not falsely claim to the OS that they aren't fully written yet, and do some internal housekeeping (journal-like) for recovery? Does SATA/SCSI support the concept of "sync", or would allowing the disk to fall behind without deceiving the OS kill performance in some way?
- gvb 13y ago
  Inside the SSD are flash memory chips. It takes a relatively long time (typically 10mSec) to tunnel the charge that changes a '1' bit (the erased state) to a '0' bit. If the power goes away during that programming time, the result in indeterminate. Not just bad, but bad in a potentially unknown way. The worst case, which I've experienced with direct writing to flash chips, is that a totally unexpected flash location is corrupted. My guess was that the CPU started the write cycle just as power was lost, the CPU glitched the address lines as it was losing its brains, and the flash corrupted a random location. Very, very bad. If a power loss causes the flash to scribble on the wrong SSD location (e.g. the tables that keep track of good and bad blocks), the SSD "dies".
  - zAy0LfpBZLC8mAC 13y ago
    That's still no reason why you would need "power loss protection", in the sense of energy storage. What is needed is proper brown-out detection and properly set up reset circuitry so that the write gets aborted before lines start glitching (that is to say: energy needs to be dumped so it can not cause any damage once the CPU starts losing its brains). That the storage location where a write was in progress is indeterminate afterwards shouldn't matter - between the time that some software initiates a write() and the time an fsync() on the same file returns, there is no guarantee what the written location will contain after a power failure, and if your software relies on the value in any way whatsoever, your software is broken.
    - brianpgordon 13y ago
      10 milliseconds is far too long to wait for a write in application code.
colin_mccabe 13y ago
It is good to see someone actually testing the power-loss protection claims made by manufacturers. However, uninterruptible power supplies are usually a better investment than power-loss resilient storage media. The problem is, even if your SSD or hard drive behaves perfectly during a power-loss scenario, your server software may not. Almost every database, filesystem, etc. includes some amount of buffering in memory, because sending every write directly to disk is a performance killer. Also, the best-case scenario with power-loss resilient media is that your system shuts down cleanly. With a UPS, you can keep the system up until diesel generators kick in, a much better endgame for everybody. I once asked someone who had worked in the hard drive business what a hard drive would do when power was lost. "Try to park the drive head immediately before it crashes on to the platter," was the immediate response. Trying to flush the cache contents wasn't even remotely on his mind. In practice, losing power while writing to a hard disk does often corrupt sectors-- even sectors that weren't being written to during the power loss incident. It's good to see that (some) SSDs are at least trying to flush the cache, but you really have to ask yourself: can you really trust the manufacturer's claims? And if you can trust them, can you trust your specific software configuration under this unusual scenario? I think it's just too long a frontier to guard with too few sheriffs. Dude, you're getting a UPS.
- miahi 13y ago
  The problem of SSDs, as others have mentioned here, is that writing during power loss can have more important effects: it can overwrite firmware bits or mapping bits. In the first case the whole SSD is dead, in the second case, much more data than the one currently written is lost.
  - colin_mccabe 13y ago
    My impression was that even cheap SSDs should have ultracapacitors or small batteries that allow them to survive a power loss event without being bricked. Of course, the stuff in the cache is lost at that point, but that's no worse than the situation with a hard drive. Also, as I mentioned, "much more data than the one currently written" can be lost when power fails in a hard drive. So the situation is really no different, unless the manufacturer screwed up.
PhantomGremlin 13y ago
In general, Intel is a "class act". A few years ago my employer worked with Intel on a joint IC project (not Flash). My overall impression was that Intel engineers were meticulous and smart. This internal culture probably applies to many different Intel divisions. So I'm not surprised that Intel SSDs are reliable.
- zerohp 13y ago
  I had an old Intel G1 SSD die during a power outage that went a little too long for my cheap UPS. Of course one event in uncontrolled conditions isn't meaningful. I still bought another Intel SSD because every Intel device I've owned was top notch.
- kozlovsky 13y ago
  Yesterday my Intel SSD 320 failed horrible and kills all data on my RAID system. I still do not understand how it happened. I was sure my storage system is very safe. My system was configured as a mirror RAID with two 1TB HDDs. The RAID was created with Intel Rapid Storage technology (my motherboard is Asus p8z77v-pro). The failed SSD was used as a RAID cache configured with Intel Smart Response technology. I was wary to use SSD directly as a main system drive, because I heard of "BAD_CTX 13F" error which happened with Intel 320 drives. My hope was that if SSD is used as just a cache, then in the case of SSD failure the data still be safe. Since this error reportedly occurs during power outage only, I set up an UPS. But all these precautions have not helped. Yesterday I surfed web with Google Chrome, and my computer suddenly become unresponsive. At first only Chrome was unusually slow, and other open programs work normally, but in a few minutes the computer was totally freeze. I was forced to press reset, and upon restart Windows automatically entered into non-interruptible "recovery mode". After more then 24 hours the OS reported that "further recovery is impossible" and the RAID become unbootable. The SSD serial number was changed to "BAD_CTX 0000013F", the sign of famous "8mb bug". It is interesting that in my case this bug was not caused by any power outage except when I pressed "reset" button, but I don't think this is count as a power loss. I take an HDD out of the RAID to connect it to other computer and save critical data, but without any success. At first sight all file system looks correct, and I even manage to copy all recent data files, but when I looked into those files it was total mess. Each file consist of some arbitrary chunks of unrelated files, mixed in random order - a bit of some executable file, several lines of my project source code, followed by chunk of some unknown xml configuration file, followed by random bytes, etc. A total mess. I still don't understand the reason of such spectacular data corruption. I have three hypotheses: 1. SSD cache sent incorrect data to RAID on write (two month ago I switched SSD cache from "enhanced" to "maximized" mode, in which writes initially goes to SSD and only then to the RAID disks). 2. Intel RAID controller goes crazy due to a program error. 3. Windows corrupt data during non-interruptible "recovery" phase. The moral is, even Intel SSD with UPS is not safe, and mirror RAID cannot not protect data from such errors.
MatthewElvey 13y ago
I have a Viking VRFS21100GBCNS which supposedly features Super Capacitor power failure protection. Manufacturer PDF - https://docs.google.com/file/d/0B9JXvW974L4JUkxJMGJIUjNtRDg/edit?usp=sharing https://docs.google.com/file/d/0B9JXvW974L4JUkxJMGJIUjNtRDg/...
ricardobeat 13y ago
Would be interesting to know how Samsung SSDs, very popular now and used in Apple's line, fare on that test.
rythie 13y ago
I've found that the Intel 320 drives are much than the Samsung 830/840, Crucial M500 when setup for sync writes on Linux (as a NFS server) - with the Intel drives being about 4x faster than the others in my testing.
ghshephard 13y ago
I read through the report a couple times, but couldn't figure out how many drives he had in each test batch. I'm a little concerned that he just took a single drive, and hoped it would effectively represent the entire model. Clearly there will be some variability - and he may have gotten a good, or bad drive - and a larger population of SSDs might behave quite differently in terms of reliability.
arh68 13y ago
Where are all the dates in this article? When were these drives purchased/manufactured? Some (all?) of the 320 drives (pre-2012) had a bug that basically bricked the drive after power loss. See more on google at '8mb bug' and this Intel thread [1]. The existence of this bug, the well-known reliability reputation of Intel, and the sheer size of this sampling number (N=500?) make the distinction in time important. Were all these drives more recent, or is the Intel failure rate, even with buggy firmware, still ~.5% ? >However, given that deployment of over 500 Intel 320 SSDs has been carried out and only 3 failures observed over several years, it would be reasonable to conclude that Intel S3500s could be trusted long-term as well [1] https://communities.intel.com/message/133499 https://communities.intel.com/message/133499
- nexox 13y ago
  That bug was not as prominent as you make it sound - I was unable to reproduce the issue (or any other issue) across tens of thousands of reboots on the buggy firmware (running Linux and a quality SAS HBA.) The circumstances to produce a corruption were much more rare than simply "power loss," and many users with "safe" platforms could easily expect 0.5% AFR. Even with "risky" OS and controller combinations, there was an element of probability involved, so most (probably almost all) power loss events would not hit the bug. Plus the SSD320 was difficult to obtain back then, and reasonable operators upgraded to the firmware version with this bug fixed, so only a small percentage of the units were ever even vulnerable.

168 points by yapcguy 13y ago

The author of this study submitted it to Slashdot with the following summary: "After the reports on SSD reliability and after experiencing a costly 50% failure rate on over 200 remote-deployed OCZ Vertex SSDs, a degree of paranoia set in where I work. I was asked to carry out SSD analysis with some very specific criteria: budget below £100, size greater than 16Gbytes and Power-loss protection mandatory. This was almost an impossible task: after months of searching the shortlist was very short indeed. There was only one drive that survived the torturing: the Intel S3500. After more than 6,500 power-cycles over several days of heavy sustained random writes, not a single byte of data was lost. Crucial M4: failed. Toshiba THNSNH060GCS: failed. Innodisk 3MP SATA Slim: failed. OCZ: failed hard. Only the end-of-lifed Intel 320 and its newer replacement, the S3500, survived unscathed. The conclusion: if you care about data even when power could be unreliable, only buy Intel SSDs." Source: http://hardware.slashdot.org/story/13/12/27/208249/power-loss-protected-ssds-tested-only-intel-s3500-passes http://hardware.slashdot.org/story/13/12/27/208249/power-los...

colechristensen 13y ago
Did he do any control studies with spinning disks? I have my doubts that many of them would survive thousands of power cycles per day.
- AmVess 13y ago
  There's probably not much of anything that would survive that many power cycles in a day. The test is pretty pointless.
  - ak217 13y ago
    It's not pointless, just misrepresented. SSDs enable new classes of applications for high-performance, non-power-protected computers. With an SSD and Intel ULV CPU, I can put together a cheap high-performance computer with no moving parts and no batteries. That has the potential to be very robust and have an expected lifetime of decades, but only if the SSD is rock solid.
    - cwzwarich 13y ago
      Which SSD is going to last for decades in a strenuous environment?
      - ak217 13y ago
        Intel, apparently
        RyanZAG 13y ago
        This test really doesn't prove that at all. There are vastly different factors between power cycling in a single day and long term continuous use. You can't take results from one and get any meaningful information about the other.
        ak217 13y ago
        You're right. But it does show one way in which it's resilient. Given this information and what else I know about Intel SSDs, I would expect them to be the most resilient overall.
  - rlpb 13y ago
    What if I have thousands of disks deployed, and I suffer a single power loss? What are the chances that at least one disk will contain corrupt data after such an event? Instead of buying thousands of disks to test this scenario, one reasonable shortcut might be to repeatedly test a single disk. Even a spinning disk can handle a few thousand power cycles in its lifetime, surely?
    - AmVess 13y ago
      On power up, there's a pretty big spike in power usage as the controller gets its shit together. Normally, this is not an issue, since the SSD housing simply absorbs the heat and then goes about its business. The same holds true for spindle drives. What this guy did was test the drives in a manner they weren't designed for. Sure, I can drop Corvettes off a 20 story building, then bitch about the results, but that wouldn't change the fact that my test was flawed from the onset. All he did was subject something to an environment is wasn't designed for. Sure, I can drop a Corvette from a 20 story building, but there's nothing to be gained when the crumple zone is packed into the tail lights.
      - zAy0LfpBZLC8mAC 13y ago
        Hu? Where should that spike in power usage come from?
        rational_indian 13y ago
        It comes from the decoupling/filter capacitors used in the DC power circuit. When the power is turned on after a sufficiently long time these capacitors are all uncharged and appear as "shorts" to the power source thus drawing large amounts of current. This initial surge current drops off as the capacitors get charged.
        zAy0LfpBZLC8mAC 13y ago
        Filter capacitors producing excessive heat due to inrush current? That would be bad filter capacitors indeed ;-) The claim was that an SSD somehow converted more electric energy into heat immediately after power-up which would damage the SSD, so real consumption, not just a current peak that goes into storage for later consumption. Normal-ESR electrolytics might have a heat problem when used at a few kHz in switching applications, but certainly not at 0.1 Hz.
  - ytjohn 13y ago
    From what they said about these reading sensor data, I'm picturing a factory, industrial, or field type scenario. I'm picturing a very small embedded computer with some sensors attached and an internal ssd. Referencing a 1.5gb OS like they did makes me think they just have a normal OS installed, but it's entirely possible that they have a modified kernel and fast boot process. Essentially, it sounds like they have a single unit that gets turned on, gathers sensor readings, and then get turned off. I'm guessing these are not handheld devices, otherwise they most likely would have a battery attached. If a non 24/7 factory, these could be turned off with the rest of the assembly line by someone throwing breakers off on the way out (very common).
- nexox 13y ago
  I've seen a few different models of 2.5" Enterprise SAS drives go through a similar test (as a control,) and they were perfectly fine with no corruption after around 2 weeks of reboots at approximately 10 minute intervals. It's the SSDs that I worry about - some store their own firmware on the same NAND as the User's data, and allow it to become corrupted during power-loss events. Several cheap models won't last more than a couple days before they simply drop off the SATA bus and never come back.

Mithaldu 13y ago

Terrible title, terrible science, terrible conclusion. Spoiler: He only tested these five drives, only Intel survived, so if they are your candidates, apply his conclusion: Crucial M4 Toshiba THNSNH060GCS 60gb Innodisk 3MP Sata Slim OCZ Vertex 32gb Intel 320 and S3500 Notably missing is the Intel runner-up Samsung and probably others i'm not aware of, as well as other models.

randall 13y ago
A little harsh, don't you think? I'd say it's still useful to have 5 data points than 0.
- Mithaldu 13y ago
  I'd have been gentle if his conclusion had included a measure of humbleness and at least kind of approached being somewhat accurate. :)
  - samplonius 13y ago
    And where was the "terrible science"? Requiring "humbleness" in posts seems to only apply to other people's posts, apparently.
    - emn13 13y ago
      The conclusion "buy intel" makes no sense. He tested far too few drives to make that conclusion, and many potentially safe alternatives went untested. Not to mention the fact that not all intel drives apparently are safe. So yeah; it's unfortunately bogus science - he's drawing invalid conclusions from a far too small sample.
    - vacri 13y ago
      Terrible science = small, restricted subject population, from which an ironclad blanket conclusion is made, that covers the entire population. One of the restrictions is price, which makes little sense in concluding so strongly for a measure involving quality. Not to mention that only a single specimen of a single model of each brand was tested. The top-voted comment in the thread points out some Intel models that don't have supercapacitors in them. Already the article is updated with a couple of other drives to test. I guess that wasn't "End of discussion" after all...

maerF0x0 13y ago

As many have pointed out: There are many applications where power cycling is extremely rare and also not a big deal. 1. Laptops have a built in UPS incase they're unplugged 2. Servers should have UPS incase they're unplugged or small outtages 3. Desktop drives shouldnt be trusted as the only copy. Though I imagine the data corruption would propagate to backups?

MichaelGG 13y ago
Unless the server UPS is inside the chassis, disk corruption on power loss seems like a very high risk. Datacenter-wide UPSes can fail (at one of XO's DCs, we had power loss 3 times in 2 months). People can accidentally unplug stuff.
JohnTHaller 13y ago
Servers can go down more than you think. Rackspace had an entire wing of their Texas data center go down hard 3 times within a couple months a few years back. I know because PortableApps.com's dedicated box was in that wing. It was an issue with the equipment that switches form external power to battery to generator power. And there was no secondary backup to that, so it would go down and take the whole wing all at once. This would corrupt our Drupal database and it would take about 8 hours to rebuild. So, having a reliable SSD with power loss prevention could be a lot more relevant than you expect.
mike_esspe 13y ago
Laptops are vulnerable to this problem, if you don't know about it. On my notebook I've lost a partition with Crucial M4. OS hanged, I did a hard reset and after reboot discovered data loss.
- userbinator 13y ago
  I got an Intel SSD (G1 80GB) in my laptop to do OS development (drivers), which incidentally is a use case that requires lots of hard resets. Never had any problems with it, and it's been through dozens if not hundreds of hard reset cycles. In fact I got the SSD specifically because I didn't want to be spinning up/down a disk that much, and laptops tend not to have reset buttons (they really need one, IMHO.)

_mikz 13y ago

Would be nice to see this test for Samsung drivers which should be closer to Intel's quality.

CamperBob2 13y ago

It's long been conventional wisdom that you'd have to be crazy to buy anything but Intel when it comes to SSDs. This study isn't too surprising, in that regard.

joenathan 13y ago
I've never had any issues with Samsung or Kingston SSDs. The 840 EVO series and Pro series are the best performance for the money you can get.
- yapcguy 13y ago
  Stick to the Pro. The EVO are based on TLC which are cheaper to manufacture and have a reduced life-span.
- userbinator 13y ago
  Kingston used to rebrand Intel SSDs (same controller, just a smaller amouht of flash) for the "value market", so not surprising they were good.
- rythie 13y ago
  The Samsung drives are poor when used on a NFS server with sync enabled - they are about 4x slower than the Intel 320 drives doing that. When used without sync, the same if no faster than the Intel drives - with the drawback of course that you've probably got corrupt data if you crash or lose power.
kevinchen 13y ago
I think it's more "buy only Intel if you have crazy reliability requirements and no periodic backups." For a typical user who just wants their stuff to load faster, it doesn't really matter: laptops shut down gracefully when they run out of power and power outages don't happen that often.
- robocat 13y ago
  We only buy Intel for staff because: their time is worth a lot, any interruption due to a flaky drive can be expensive (especially if traveling), and to reduce worry. The time for me to research whether another Brand is reliable costs more than just choosing Intel. A no brainer decision. I.e. the savings on a cheaper drive are not worth the risks. Backup, always, of course.

deleted 13y ago

[deleted]

herf 13y ago

The Crucial M4 had no power-loss protection. The newer model (M500) does and would be a more interesting test.

slyall 13y ago

I really hate when people are sloppy with notation. 20GB = Twenty Gigabytes NOT 20gb = Twenty gram bits and: 20MB/s = Twenty Megabytes per second NOT 20mbytes/sec = Twenty milli-bytes per second ( If you got you B's and b's correct about you would need to write out "bytes" )

jpk 13y ago
My favorite is "5KB". The prefix for kilo- is a lowercase 'k'. Uppercase 'K' is Kelvin. I don't even know how to conceptualize a Kelvin-Byte.
- wyager 13y ago
  >I don't even know how to conceptualize a Kelvin-Byte. I can actually see where this unit would be useful; in determining the probability of bit rot. The more kelvins you have, and the more bytes you have, the more likely you are to flip a bit due to random fluctuations. temperature*storage capacity = Kelvin-Bytes.
- zAy0LfpBZLC8mAC 13y ago
  Once you have that figured out, try to conceptualize Kelvinbyteohms aka Kelvinbytes per Siemens!
illicium 13y ago
If you want to be really pedantic, differentiate SI giga/megabytes as GB/MB and "binary" gibi/mebibytes as GiB/MiB
- StavrosK 13y ago
  So 1024 bytes is 1 kiB?
  - jpk 13y ago
    Yep, see: http://en.wikipedia.org/wiki/Binary_prefix http://en.wikipedia.org/wiki/Binary_prefix

mandeepj 13y ago

My brand new crucial M4 ssd stopped working after few days. To be precise, the system would show the drive for about 2 minutes after each reboot then after that the drive would not show up anywhere at all. Whether you try in windows explorer or device manager or any where else. I contacted their support team but they never replied.

sitkack 13y ago
Crucial reportedly has better support than that. My M4 would crash after 21 minutes (exactly-ish) over a year continuous service (100+GB/day), the latest firmware fixed the problem and increased speeds. The USB firmware updater was a god send compared to the vile vile hoops I had to jump through to upgrade my OCZ drive. I currently have an Samsung 840 1TB and has been rock solid, it replaced a 250GB intel 320. Afaik Samsung is the only manufacturer that owns the whole supply chain, flash + controller.
- kalleboo 13y ago
  Toshiba also do both NAND and controllers.
codecop 13y ago
I got my Crucial M4 4 month ago and it stopped working too. BIOS dont show it. Got replaced with new one. After 3-4weeks failed again. Bios shows SSD but testing shows many errors. I will send it again or will try update firware.

strlen 13y ago

What I find rather annoying is that manufacturers/marketers rarely state whether or not an SSD has a supercapacitor or not. Here have been my finding: - OCZ Vertex 4 SSD does (but this is not advertised) - OCZ Vertex 3 does not, Vertex 3 Pro does - Ditto for Vertex 2 / Vertex 2 Pro - OCZ Deneva(2) R does, other OCZ Deneva/Deneva 2 do not -- and in the case of Deneva 2 R this is advertized - Intel 320 does (but is only sata II), but this is not advertised at all - Intel 520/530 does not - Intel 330/335: unclear - Intel S3700/S3500: does and this is advertised Background: I was looking for an SSD to use for a ZFS intent log ("ZIL" -- ZFS's write ahead log) -- my requirements were a sandforce controller (or equivalent) and toggle NAND (so that I could use the same disk for both ZFS cache and the ZIL), and a supercap. This was surprisingly hard to find. What I'd like to see is: 1) Data on which SSDs have supercap 2) Data on which SSDs actually honour cache flush request (then a UPS + forcing a cache flush request upon power failure + redundant power supplies/multiple replicas in a distributed system) would suffice. 3) Best yet: have an API for for check whether or not an SSD has a supercap, whether or not the supercap still holds charge, and policy for honouring cache flush requests. Let the OS decide based on a policy I set. If you are a consumer, the practical recommendation is not very different from the practical recommendation I'd give to anyone using spinning disks: RAID1/1+0/RAIDZ(-2) (with SSDs coming from different patches so that they do not wear out at the same time), UPS (for power outages), backups (against yourself and against power supply failures). For production: obviously use a UPS, put the WAL of your database on an SSD with a supercap, make sure that your database fsync()'s the WAL at a reasonable interval (on every transaction is probably unreasonable, but so is once an hour), and use a distributed system that replicates the WAL. If using a distributed system with (semi-)synchronous WAL replication is not an option and losing incremental data is not acceptable, use redundant power supplies.

AnthonyMouse 13y ago
> If using a distributed system with (semi-)synchronous WAL replication is not an option and losing incremental data is not acceptable, use redundant power supplies. Also be sure to plug each redundant power supply into a separate UPS, and avoid equipment that has redundant power supplies on a single feed. UPS failures are a bear.

rlpb 13y ago

Why must SSDs have "power loss protection" in the form of a battery or supercapacitors to finish writes? Can they not simply cache writes but not falsely claim to the OS that they aren't fully written yet, and do some internal housekeeping (journal-like) for recovery? Does SATA/SCSI support the concept of "sync", or would allowing the disk to fall behind without deceiving the OS kill performance in some way?

gvb 13y ago
Inside the SSD are flash memory chips. It takes a relatively long time (typically 10mSec) to tunnel the charge that changes a '1' bit (the erased state) to a '0' bit. If the power goes away during that programming time, the result in indeterminate. Not just bad, but bad in a potentially unknown way. The worst case, which I've experienced with direct writing to flash chips, is that a totally unexpected flash location is corrupted. My guess was that the CPU started the write cycle just as power was lost, the CPU glitched the address lines as it was losing its brains, and the flash corrupted a random location. Very, very bad. If a power loss causes the flash to scribble on the wrong SSD location (e.g. the tables that keep track of good and bad blocks), the SSD "dies".
- zAy0LfpBZLC8mAC 13y ago
  That's still no reason why you would need "power loss protection", in the sense of energy storage. What is needed is proper brown-out detection and properly set up reset circuitry so that the write gets aborted before lines start glitching (that is to say: energy needs to be dumped so it can not cause any damage once the CPU starts losing its brains). That the storage location where a write was in progress is indeterminate afterwards shouldn't matter - between the time that some software initiates a write() and the time an fsync() on the same file returns, there is no guarantee what the written location will contain after a power failure, and if your software relies on the value in any way whatsoever, your software is broken.
  - brianpgordon 13y ago
    10 milliseconds is far too long to wait for a write in application code.

colin_mccabe 13y ago

It is good to see someone actually testing the power-loss protection claims made by manufacturers. However, uninterruptible power supplies are usually a better investment than power-loss resilient storage media. The problem is, even if your SSD or hard drive behaves perfectly during a power-loss scenario, your server software may not. Almost every database, filesystem, etc. includes some amount of buffering in memory, because sending every write directly to disk is a performance killer. Also, the best-case scenario with power-loss resilient media is that your system shuts down cleanly. With a UPS, you can keep the system up until diesel generators kick in, a much better endgame for everybody. I once asked someone who had worked in the hard drive business what a hard drive would do when power was lost. "Try to park the drive head immediately before it crashes on to the platter," was the immediate response. Trying to flush the cache contents wasn't even remotely on his mind. In practice, losing power while writing to a hard disk does often corrupt sectors-- even sectors that weren't being written to during the power loss incident. It's good to see that (some) SSDs are at least trying to flush the cache, but you really have to ask yourself: can you really trust the manufacturer's claims? And if you can trust them, can you trust your specific software configuration under this unusual scenario? I think it's just too long a frontier to guard with too few sheriffs. Dude, you're getting a UPS.

miahi 13y ago
The problem of SSDs, as others have mentioned here, is that writing during power loss can have more important effects: it can overwrite firmware bits or mapping bits. In the first case the whole SSD is dead, in the second case, much more data than the one currently written is lost.
- colin_mccabe 13y ago
  My impression was that even cheap SSDs should have ultracapacitors or small batteries that allow them to survive a power loss event without being bricked. Of course, the stuff in the cache is lost at that point, but that's no worse than the situation with a hard drive. Also, as I mentioned, "much more data than the one currently written" can be lost when power fails in a hard drive. So the situation is really no different, unless the manufacturer screwed up.

PhantomGremlin 13y ago

In general, Intel is a "class act". A few years ago my employer worked with Intel on a joint IC project (not Flash). My overall impression was that Intel engineers were meticulous and smart. This internal culture probably applies to many different Intel divisions. So I'm not surprised that Intel SSDs are reliable.

zerohp 13y ago
I had an old Intel G1 SSD die during a power outage that went a little too long for my cheap UPS. Of course one event in uncontrolled conditions isn't meaningful. I still bought another Intel SSD because every Intel device I've owned was top notch.
kozlovsky 13y ago
Yesterday my Intel SSD 320 failed horrible and kills all data on my RAID system. I still do not understand how it happened. I was sure my storage system is very safe. My system was configured as a mirror RAID with two 1TB HDDs. The RAID was created with Intel Rapid Storage technology (my motherboard is Asus p8z77v-pro). The failed SSD was used as a RAID cache configured with Intel Smart Response technology. I was wary to use SSD directly as a main system drive, because I heard of "BAD_CTX 13F" error which happened with Intel 320 drives. My hope was that if SSD is used as just a cache, then in the case of SSD failure the data still be safe. Since this error reportedly occurs during power outage only, I set up an UPS. But all these precautions have not helped. Yesterday I surfed web with Google Chrome, and my computer suddenly become unresponsive. At first only Chrome was unusually slow, and other open programs work normally, but in a few minutes the computer was totally freeze. I was forced to press reset, and upon restart Windows automatically entered into non-interruptible "recovery mode". After more then 24 hours the OS reported that "further recovery is impossible" and the RAID become unbootable. The SSD serial number was changed to "BAD_CTX 0000013F", the sign of famous "8mb bug". It is interesting that in my case this bug was not caused by any power outage except when I pressed "reset" button, but I don't think this is count as a power loss. I take an HDD out of the RAID to connect it to other computer and save critical data, but without any success. At first sight all file system looks correct, and I even manage to copy all recent data files, but when I looked into those files it was total mess. Each file consist of some arbitrary chunks of unrelated files, mixed in random order - a bit of some executable file, several lines of my project source code, followed by chunk of some unknown xml configuration file, followed by random bytes, etc. A total mess. I still don't understand the reason of such spectacular data corruption. I have three hypotheses: 1. SSD cache sent incorrect data to RAID on write (two month ago I switched SSD cache from "enhanced" to "maximized" mode, in which writes initially goes to SSD and only then to the RAID disks). 2. Intel RAID controller goes crazy due to a program error. 3. Windows corrupt data during non-interruptible "recovery" phase. The moral is, even Intel SSD with UPS is not safe, and mirror RAID cannot not protect data from such errors.

MatthewElvey 13y ago

I have a Viking VRFS21100GBCNS which supposedly features Super Capacitor power failure protection. Manufacturer PDF - https://docs.google.com/file/d/0B9JXvW974L4JUkxJMGJIUjNtRDg/edit?usp=sharing https://docs.google.com/file/d/0B9JXvW974L4JUkxJMGJIUjNtRDg/...

ricardobeat 13y ago

Would be interesting to know how Samsung SSDs, very popular now and used in Apple's line, fare on that test.

rythie 13y ago

I've found that the Intel 320 drives are much than the Samsung 830/840, Crucial M500 when setup for sync writes on Linux (as a NFS server) - with the Intel drives being about 4x faster than the others in my testing.

ghshephard 13y ago

I read through the report a couple times, but couldn't figure out how many drives he had in each test batch. I'm a little concerned that he just took a single drive, and hoped it would effectively represent the entire model. Clearly there will be some variability - and he may have gotten a good, or bad drive - and a larger population of SSDs might behave quite differently in terms of reliability.

arh68 13y ago

Where are all the dates in this article? When were these drives purchased/manufactured? Some (all?) of the 320 drives (pre-2012) had a bug that basically bricked the drive after power loss. See more on google at '8mb bug' and this Intel thread [1]. The existence of this bug, the well-known reliability reputation of Intel, and the sheer size of this sampling number (N=500?) make the distinction in time important. Were all these drives more recent, or is the Intel failure rate, even with buggy firmware, still ~.5% ? >However, given that deployment of over 500 Intel 320 SSDs has been carried out and only 3 failures observed over several years, it would be reasonable to conclude that Intel S3500s could be trusted long-term as well [1] https://communities.intel.com/message/133499 https://communities.intel.com/message/133499

nexox 13y ago
That bug was not as prominent as you make it sound - I was unable to reproduce the issue (or any other issue) across tens of thousands of reboots on the buggy firmware (running Linux and a quality SAS HBA.) The circumstances to produce a corruption were much more rare than simply "power loss," and many users with "safe" platforms could easily expect 0.5% AFR. Even with "risky" OS and controller combinations, there was an element of probability involved, so most (probably almost all) power loss events would not hit the bug. Plus the SSD320 was difficult to obtain back then, and reasonable operators upgraded to the firmware version with this bug fixed, so only a small percentage of the units were ever even vulnerable.