10 ms·

What we learned about SSDs in 2015

179 points by NN88 11y ago

transfire 11y ago
Not so sure. There are plenty of benefits to SSD too. I suspect system designers will just add more RAM to act as cache to offset some of these performance issues. Not to mention further improve temperature control.
- stefantalpalaru 11y ago
  More RAM means more reserve power needed to flush it to permanent storage when the main power is cut. What's more likely to happen is exposing the low level storage and software to kernel drivers.
  - scurvy 11y ago
    I think transfire was referring to RAM in the system to act as a pagefile read cache, not RAM on the SSD to act as a cache there. There's no power risk to an OS-level read cache.
exabrial 11y ago
Does NVMe solve the garbage collection problem?
- pkaye 11y ago
  NVMe is just the communications protocol between the host and device. Garbage collection is due to hiding the non ideal properties of the NAND from the host. Primary among them is endurance limits and the size difference between write and erase units. You could for example move the handling of some of these GC details to the OS filesystem level but then that becomes more complicated and has to deal with the differences with each NAND technology generation. You couldn't just copy a file system image from one drive to another for example.
  - nly 11y ago
    > You could for example move the handling of some of these GC details to the OS filesystem level Isn't this basically the Linux 'discard' mount flag? Most distros seem to be recommending periodic fstrim for consumer uses, what's the best practice in the data center?
    - pkaye 11y ago
      The trim is kind of like the free counterpart to malloc. When the drive has been fully written, there is a lot less free space which constraints the GC and makes it works much harder. The trim tells the drive to free up space allowing for less GC work. A big difference between client and enterprise drives is the amount of over-provisioning. A simple trick if you can't use trim is to leave 5-10% of the drive unused to improve the effective over-provisioning and improve worst case performance. The question of using trim in data centers might be due to interaction between trim and raid configurations because sometimes trim is implemented as nondeterministic (as a hint to the drive; persistant trim can cause performance drops unless you are willing to through in extra hardware to optimize it well) which causes parity calculation issues when recovering from a failed drive.
    - wtallis 11y ago
      Discard operations only tell the drive that a LBA is eligible for GC. It's basically a delete operation that explicitly can be deferred. It does not give the OS any input into when or how GC is done, and it doesn't give the OS any way to observe any details about the GC process. I think the recommendations for periodic fstrim of free space is due to filesystems usually not taking the time to issue a large number of discard operations when you delete a bunch of data. Even though discards should be faster than a synchronous erase command, not issuing any command to the drive is faster still.
      - pkaye 11y ago
        SATA drives until recently didn't have queued trims so if you did an occasional trim between read/writes you would have to flush the queue. Queued trims were added later on but have been slow to be adopted because it can be difficult to get it working fast, efficient and correct when intermingled with reads and writes. I know atleast one recent drive with queued trim had some bugs in the implementation.
_ak 11y ago
At one of my previous employers, they built a massive "cloud" storage system. The underlying file system was ZFS, which was configured to put its write logs onto an SSD. With the write load on the system, the servers burnt through an SSD in about a year, i.e. most SSDs started failing after about a year. The hardware vendors knew how far you could push SSDs, and thus refused to give any warranty. All the major SSD vendors told us SSDs are strictly considered wearing parts. That was back in 2012 or 2013.
- justin66 11y ago
  If your SSDs were wearing out after a year, and were warrantied for a year, I'm guessing you weren't using "enterprise" SSDs?
  - skrause 11y ago
    Even enterprise SSDs like Samsung's have a guarantee like "10 years or up to x TB of written data". So if you write a lot you can lose the guarantee after a year even with enterprise SSDs.
  - _ak 11y ago
    They were enterprise models (i.e. not cheap), but they had no warranty in the first place. Every single hardware supplier simply refused to give any. I _guess_ because of the expected wear and tear.
    - justin66 11y ago
      That's interesting. I don't even know how to buy these things without a warranty. Were they direct from the manufacturer?
- rsync 11y ago
  Just a note ... we use SSDs as write cache in ZFS at rsync.net and although you should be able to withstand a SLOG failure, we don't want to deal with it so we mirror them. My personal insight, and I think this should be a best practice, is that if you mirror something like an SLOG, you should source two entirely different SSD models - either the newest intel and the newest samsung, or perhaps previous generation intel and current generation intel. The point is, if you put the two SSDs into operation at the exact same time, they will experience the exact same lifecycle and (in my opinion) could potentially fail exactly simultaneously. There's no "jitter" - they're not failing for physical reasons, they are failing for logical reasons ... and the logic could be identical for both members of the mirror...
ComputerGuru 11y ago
Kind of stupid to end with "Since CPUs aren't getting faster, making storage faster is a big help." CPUs and storage exist for completely disjoint purposes, and the fastest CPU in the world can't make up for a slow disk (or vice versa). Anyway, CPUs are still "faster" than SSDs, whatever that means, if you wish to somehow compare apples to oranges. That's why even with NVMe if you are dealing with compressible data enabling block compression in your FS can speed up your IO workflow.
- planckscnst 11y ago
  Your process spends some amount of time waiting on various resources; some of those may be CPU-bound, some may be disk-bound. Speeding either of those up will make your process faster. In fact, you can even trade one for the other if you use compression.
- nly 11y ago
  Storage and CPU cycles aren't completely disjoint. While this is true for plain old data archival, a lot of storage in reality is just used as cache. You could argue even your customer data is a cache, because you can always go back to the customer for most of it. Most data can be recomposed from external sources given enough computation.
- TeMPOraL 11y ago
  Ever tried to play a modern computer game? You never have enough RAM for stuff; a lot of content gets dumped onto hard drive sooner or later (virtual memory), or is be streamed from the drive in the first place. Having faster access helps tremendously. From my observation, actually most personal and business use machines are IO-bound - it often takes just the web browser itself - with webdevs pumping out sites filled with superfluous bullshit - to fill out your RAM completely, and then you have swapping back and forth.
  - arielweisberg 11y ago
    For years total available RAM could easily exceed the install size of entire and entire game thanks to consoles holding everything back. And by exceeded I mean the games were 32-bit binaries so the ram left over was enough to cache the entire game data set even in light of RAM used by the game. Recently install size seems to have grown quite a bit.
vbezhenar 11y ago
May be SSD should add something like "raw mode", when controller just reports everything he knows about disk, and operating system takes in control the disk, so firmware won't cause unexpected pauses. After all, operating system knows more, what files are not likely to be touched, what files are changing often, etc.
- seibelj 11y ago
  And now we must rely on each OS to implement their own version of a virtual firmware, and do comparisons between different implementations, etc. etc. etc.
- wtallis 11y ago
  The industry is moving toward a mid-point of having the flash translation layer still implemented on the drive so that it can present a normal block device interface, but exposing enough details that the OS can have a better idea of whether garbage collection is urgently needed: http://anandtech.com/show/9720/ocz-announces-first-sata-host-managed-ssd-saber-1000-hms http://anandtech.com/show/9720/ocz-announces-first-sata-host... Moving the FTL entirely onto the CPU throws compatibility out the window; you can no longer access the drive from more than one operating system, and UEFI counts. You'll also need to frequently re-write the FTL to support new flash interfaces.
  - nqzero 11y ago
    OS compatibility is important for laptops/desktops, but not in at least some database / server applications, and those are the applications that would benefit most from raw access
- TD-Linux 11y ago
  This has existed for a long time, see the Linux kernel's mtd infrastructure, and the filesystems designed to run on top of it (jffs, yaffs). It used to be used in a lot of embedded devices before eMMC became cheap, and is still used in things like home routers. I am not sure how well mtd's abstraction fits with modern Flash chips, though.
- deleted 11y ago
japaw 11y ago
"2015 was the beginning of the end for SSDs in the data center." is quit a bold statement especial when not discussing any alternative. I do not see us going back to magnetic disk, and most new storage technology are some kind of ssd...
- seabrookmx 11y ago
  Yeah I thought that too. I think the author is specifically referring to 2.5" form factor SATA SSD's though.
- scurvy 11y ago
  My thoughts exactly. The article is quite inflammatory and tosses out some bold statements without really deep diving into them. My favorite: "Finally, the unpredictable latency of SSD-based arrays - often called all-flash arrays - is gaining mind share. The problem: if there are too many writes for an SSD to keep up with, reads have to wait for writes to complete - which can be many milliseconds. Reads taking as long as writes? That's not the performance customers think they are buying." This is completely false in a properly designed server system. Use the deadline scheduler with SSD's so that reads aren't starved from bulk I/O operations. This is fairly common knowledge. Also, if you're throwing too much I/O load at any storage system, things are going to slow down. This should not be a surprise. SSD's are sorta magical (Artur), but they're not pure magic. They can't fix everything. While Facebook started out with Fusion-io, they very quickly transitioned to their own home-designed and home-grown flash storage. I'd be wary of using any of their facts or findings and applying them to all flash storage. In short, these things could just be Facebook problems because they decided to go build their own. He also talks about the "unpredictability of all flash arrays" like the fault is 100% due to the flash. In my experience, it's usually the RAID/proprietary controller doing something unpredictable and wonky. Sometimes the drive and controller do something dumb in concert, but it's usually the controller. EDIT: It was 2-3 years ago that flash controller designers started to focus on uniform latency and performance rather than concentrating on peak performance. You can see this in the maturation of I/O latency graphs from the various Anandtech reviews.
  - fleitz 11y ago
acquacow 11y ago
...unless you are Fusion-io, in which case, most of these problems don't affect you.
- ddorian43 11y ago
  why? isn't fusion-io based on ssd ?
  - jjtheblunt 11y ago
    fusion-io (i believe, but please verify online) uses a spinning drive for frequent writes and an ssd for frequent reads, with software deciding what goes where, lessening the write traffic to the ssd, and thus wear to it.
    - acquacow 11y ago
      No, Fusion-io has nothing to do with spinning drives. They make PCI-e flash drives with an FPGA as the "controller" for the flash. There are multiple communication channels, so you can simultaneously read and write from the drives, and there are various tunings available to control how garbage collection works. They are the only "SSD" that doesn't hide all the flash behind any kind of legacy disk controller or group protocol like NVMe
    - jaysoncena 11y ago
      I think what you're trying to describe is apple's fusion drive.
  - acquacow 11y ago
    They use flash like all other SSDs, but don't use a disk controller or any community defined protocols for mass compatibility. They use their own software and an FPGA to control the flash.
  - pkaye 11y ago
    They have a special OS driver which moves the FTL closer to the OS. So one of the things they need is multi GB of memory from the OS to keep the FTL mapping tables. Also sudden power loss requires OS intervention to piece back the FTL structure (I seem to recall the original product taking 5 minutes to recover.) This also means you couldn't boot from an Fusion IO drive. I'm not sure if they fixed these issues on a more recent drive.
Hoff 11y ago
FWIW, the top-end HPE SSD models are rated for up to 25 writes of the entire SSD drive, per day, for five years. The entry-level SSDs are rated for ~two whole-drive writes per week. Wear gage, et al. http://www8.hp.com/h20195/v2/GetPDF%2Easpx%2F4AA4%2D7186ENW%2Epdf http://www8.hp.com/h20195/v2/GetPDF%2Easpx%2F4AA4%2D7186ENW%...
- tim333 11y ago
  Also maybe of interest, the Techreport The SSD Endurance Experiment. Their assorted drives lasted about 2,000 - 10,000 whole disk writes. http://techreport.com/review/27909/the-ssd-endurance-experiment-theyre-all-dead http://techreport.com/review/27909/the-ssd-endurance-experim...
velox_io 11y ago
The most exciting recent development in SSDs (until 3DXpoint is released), is bypassing the SATA interface, connecting drives straight into the PCIe bus (no more expensive raid controllers). Just a shame hardly any server motherboards come with M.2 slots right now. The 4x speed increase and lower CPU overhead means it is now possible to move RAM only applications (for instance in-memory databases) to SSDs, keeping only the indexes in memory. Yeah, we've been going that way for a while, just seems we've come a long way from expensive Sun e6500's I was working with in just over a decade ago.
- wtallis 11y ago
  M.2 slots don't make much sense for servers, at least if you're trying to take advantage of the performance benefits possible with a PCIe interface. Current M.2 drives aren't even close to saturating the PCIe 3.0 x4 link but they're severely thermally limited for sustained use and they're restricted in capacity due to lack of PCB area. Server SSDs should stick with the traditional half-height add-in card form factor with nice big heatsinks.
  - scurvy 11y ago
    2.5" NVMe is fine and more scalable than add-in cards (more slots) with increased serviceability. The equivalent add-in card is usually much more expensive than the 2.5" version.
  - lsc 11y ago
    most of the NVMe backplanes I've seen give full 'enterprise 2.5" drive clearance to the thing, so if they are actually as thick as consumer SSDs, as most current SATA 'enterprise SSD' are, there's plenty of room for a heatsink without expanding the slot. The supermicro chassis (and I've only explored the NVMe backplanes from supermicro) usually put a lot of effort into drawing air through the drives, so assuming you put in blanks and stuff, the airflow should be there, if the SSD are setup to take advantage of it.
    - wtallis 11y ago
bravura 11y ago
I need an external disk for my laptop that I leave plugged in all the time. What is the most reliable external hard drive type? I thought SSDs were more reliable than spinning disks, especially to leave plugged in constantly, but now I'm not as sure.
- Thorbears 11y ago
  This article, and the majority of the comments here, are about using SSDs in server environments, where permanently high load and zero downtime is the norm. And it doesn't even seem to be about SSDs vs HDDs, it is about SSDs vs future technologies. For personal use, SSDs outperform HDDs in just about every aspect, if you can afford the cost, an SSD is the better choice. And there is nothing mentioned here about downsides of leaving a drive plugged in and powered on at all times.
- ScottBurson 11y ago
  I still don't trust SSDs as much as I do spinning disks. While neither kind of drive should be trusted with the only copy of important data, I would say that drives used for backup, or for access to large amounts of data that can be recovered or recreated if lost and where the performance requirements do not demand an SSD, might as well be HDDs -- they're cheaper, and arguably still more reliable. If the workload is write-heavy, HDDs are definitely preferred as they will last much longer. While all disks can fail, HDDs are less likely to fail completely; usually they just start to develop bad sectors, so you may still be able to recover much of their contents. When an SSD goes, it generally goes completely (at least, so I've read). So it depends on your needs and how you plan to use the drive. For light use, it probably doesn't matter much either way. For important data, you need to keep it backed up in either case. SSDs use less power, particularly when idle, so if you're running on battery a lot, that would be a consideration as well.
  - cnvogel 11y ago
    Anecdotal evidence: 2.5" SATA HDD failed for me suddenly just last Tuesday, SMART was fine before, both the attributes and a long (complete surface scan) selftest I did a few weeks ago after I got this lightly used notebook from a colleague (I only needed for tests). I think what people experience is that the sudden death of SSDs doesn't occur more often than on HDDs. But with the mechanical issues and slowly building up of bad sectors gone, sudden death is probably the only visible issue left. (Just my personal opinion.)
ak217 11y ago
This article has a lot of good information, but its weirdly sensationalistic tone detracts from it. I appreciate learning more about 3D Xpoint and Nantero, but SSDs are not a "transitional technology" in any real sense of the word, and they won't be displaced by anything in 2016, if nothing else because it takes multiple years from volume capability to stand up a product pipeline on a new memory technology, and more years to convince the enterprise market to start deploying it. The most solid point the article makes is that the workload-specific performance of SSD-based storage is still being explored, and we need better tools for it.
- nostrademons 11y ago
  I got the sense that it was a PR hit for Nantero, bought and paid for. Notice the arc of the article: it says "[Popular hot technology] is dead. [Big vendors] have recently introduced [exciting new product], but [here are problems and doubts about those]. There's also [small startup you've never heard of] which has [alternative product] featuring [this list of features straight from their landing page]." Usually these types of articles are designed to lead people directly to the product that's paying for the article. Sensationalistic is good for this; it gets people to click, it gets people to disagree, and then the controversy spreads the article across the net. Seems like it's working, in this case.
  - Laforet 11y ago
    Robin Harris has been advocating for the abolishment of block abstraction layer for a couple of years now and this piece is consistent with his usual rhetoric
  - Dylan16807 11y ago
    Eh, it listed it as "promises" and talked the same way about Adesto. It's reasonable to say "this is the basic claim of the product; we'll see if they get there" without it being PR.
fleitz 11y ago
Given the number of IOPS SSDs produce they are a win even if you have to chuck them every 6 months.
ilaksh 11y ago
Article is garbage. Basically "I told you so" by someone who never got up-to-date after the first SSDs came out and found some numbers to cherry pick that seemed to support his false beliefs.
rodionos 11y ago
>log-structured I/O management built into SSDs is seriously sub-optimal for databases and apps that use log-structured I/O as well This assert piqued my interest given that my hands-on experience with HBase speaks to the contrary. The paper by SanDisk they refer to https://www.usenix.org/system/files/conference/inflow14/inflow14-yang.pdf https://www.usenix.org/system/files/conference/inflow14/infl... seems to suggest that most of the issues are related to sub-optimal degragmentation by the disk driver itself. More specifically, the fact that some of the defragmentation is unnecessary. Hardly a reason to blame the databases and can be addressed down the road. After all, GC in Java is still an evolving subject.
otakucode 11y ago
Can't wait for the inevitable discovery that NAND chips are being price-fixed. You would think after the exact same companies price-fixed RAM and LCD panels that peoples radars would go off faster. You expect me to believe that the metals and neodymium magnets and now helium containing drives are cheaper to manufacture than an array of identical NAND chips? NAND chips which are present in a significant percentage of all electronic products purchased by anyone anywhere? When a component is that widely used, it becomes commoditized. Which means its price to manufacture drops exponentially, not linearly like the price drops of NAND has done. This happened with RAM and LCDs as well. When you can look around and see a technology being utilized in dozens of products within your eyesight no matter where you are, and those products are still more expensive than the traditional technology they replace, price-fixing is afoot. I am open to being wrong on this, but I don't think I am. Can anyone give a plausible explanation why 4TB of NAND storage should cost more to manufacture than a 4TB mechanical hard drive does, given the materials, widespread demand for the component, etc?
- pjc50 11y ago
  "Apples do not cost the same as oranges, therefore oranges are being price-fixed" is not a convincing line of reasoning. The two technologies are very different and NAND storage is much newer, and has always been much more expensive than disk storage. The correct thing to compare NAND prices to is other chips that are being fabbed at the same process node, by die area.
unixhero 11y ago
Great thread.

179 points by NN88 11y ago

Not so sure. There are plenty of benefits to SSD too. I suspect system designers will just add more RAM to act as cache to offset some of these performance issues. Not to mention further improve temperature control.

stefantalpalaru 11y ago
More RAM means more reserve power needed to flush it to permanent storage when the main power is cut. What's more likely to happen is exposing the low level storage and software to kernel drivers.
- scurvy 11y ago
  I think transfire was referring to RAM in the system to act as a pagefile read cache, not RAM on the SSD to act as a cache there. There's no power risk to an OS-level read cache.

exabrial 11y ago

Does NVMe solve the garbage collection problem?

pkaye 11y ago
NVMe is just the communications protocol between the host and device. Garbage collection is due to hiding the non ideal properties of the NAND from the host. Primary among them is endurance limits and the size difference between write and erase units. You could for example move the handling of some of these GC details to the OS filesystem level but then that becomes more complicated and has to deal with the differences with each NAND technology generation. You couldn't just copy a file system image from one drive to another for example.
- nly 11y ago
  > You could for example move the handling of some of these GC details to the OS filesystem level Isn't this basically the Linux 'discard' mount flag? Most distros seem to be recommending periodic fstrim for consumer uses, what's the best practice in the data center?
  - pkaye 11y ago
    The trim is kind of like the free counterpart to malloc. When the drive has been fully written, there is a lot less free space which constraints the GC and makes it works much harder. The trim tells the drive to free up space allowing for less GC work. A big difference between client and enterprise drives is the amount of over-provisioning. A simple trick if you can't use trim is to leave 5-10% of the drive unused to improve the effective over-provisioning and improve worst case performance. The question of using trim in data centers might be due to interaction between trim and raid configurations because sometimes trim is implemented as nondeterministic (as a hint to the drive; persistant trim can cause performance drops unless you are willing to through in extra hardware to optimize it well) which causes parity calculation issues when recovering from a failed drive.
  - wtallis 11y ago
    Discard operations only tell the drive that a LBA is eligible for GC. It's basically a delete operation that explicitly can be deferred. It does not give the OS any input into when or how GC is done, and it doesn't give the OS any way to observe any details about the GC process. I think the recommendations for periodic fstrim of free space is due to filesystems usually not taking the time to issue a large number of discard operations when you delete a bunch of data. Even though discards should be faster than a synchronous erase command, not issuing any command to the drive is faster still.
    - pkaye 11y ago
      SATA drives until recently didn't have queued trims so if you did an occasional trim between read/writes you would have to flush the queue. Queued trims were added later on but have been slow to be adopted because it can be difficult to get it working fast, efficient and correct when intermingled with reads and writes. I know atleast one recent drive with queued trim had some bugs in the implementation.

_ak 11y ago

At one of my previous employers, they built a massive "cloud" storage system. The underlying file system was ZFS, which was configured to put its write logs onto an SSD. With the write load on the system, the servers burnt through an SSD in about a year, i.e. most SSDs started failing after about a year. The hardware vendors knew how far you could push SSDs, and thus refused to give any warranty. All the major SSD vendors told us SSDs are strictly considered wearing parts. That was back in 2012 or 2013.

justin66 11y ago
If your SSDs were wearing out after a year, and were warrantied for a year, I'm guessing you weren't using "enterprise" SSDs?
- skrause 11y ago
  Even enterprise SSDs like Samsung's have a guarantee like "10 years or up to x TB of written data". So if you write a lot you can lose the guarantee after a year even with enterprise SSDs.
- _ak 11y ago
  They were enterprise models (i.e. not cheap), but they had no warranty in the first place. Every single hardware supplier simply refused to give any. I _guess_ because of the expected wear and tear.
  - justin66 11y ago
    That's interesting. I don't even know how to buy these things without a warranty. Were they direct from the manufacturer?
rsync 11y ago
Just a note ... we use SSDs as write cache in ZFS at rsync.net and although you should be able to withstand a SLOG failure, we don't want to deal with it so we mirror them. My personal insight, and I think this should be a best practice, is that if you mirror something like an SLOG, you should source two entirely different SSD models - either the newest intel and the newest samsung, or perhaps previous generation intel and current generation intel. The point is, if you put the two SSDs into operation at the exact same time, they will experience the exact same lifecycle and (in my opinion) could potentially fail exactly simultaneously. There's no "jitter" - they're not failing for physical reasons, they are failing for logical reasons ... and the logic could be identical for both members of the mirror...

ComputerGuru 11y ago

Kind of stupid to end with "Since CPUs aren't getting faster, making storage faster is a big help." CPUs and storage exist for completely disjoint purposes, and the fastest CPU in the world can't make up for a slow disk (or vice versa). Anyway, CPUs are still "faster" than SSDs, whatever that means, if you wish to somehow compare apples to oranges. That's why even with NVMe if you are dealing with compressible data enabling block compression in your FS can speed up your IO workflow.

planckscnst 11y ago
Your process spends some amount of time waiting on various resources; some of those may be CPU-bound, some may be disk-bound. Speeding either of those up will make your process faster. In fact, you can even trade one for the other if you use compression.
nly 11y ago
Storage and CPU cycles aren't completely disjoint. While this is true for plain old data archival, a lot of storage in reality is just used as cache. You could argue even your customer data is a cache, because you can always go back to the customer for most of it. Most data can be recomposed from external sources given enough computation.
TeMPOraL 11y ago
Ever tried to play a modern computer game? You never have enough RAM for stuff; a lot of content gets dumped onto hard drive sooner or later (virtual memory), or is be streamed from the drive in the first place. Having faster access helps tremendously. From my observation, actually most personal and business use machines are IO-bound - it often takes just the web browser itself - with webdevs pumping out sites filled with superfluous bullshit - to fill out your RAM completely, and then you have swapping back and forth.
- arielweisberg 11y ago
  For years total available RAM could easily exceed the install size of entire and entire game thanks to consoles holding everything back. And by exceeded I mean the games were 32-bit binaries so the ram left over was enough to cache the entire game data set even in light of RAM used by the game. Recently install size seems to have grown quite a bit.

vbezhenar 11y ago

May be SSD should add something like "raw mode", when controller just reports everything he knows about disk, and operating system takes in control the disk, so firmware won't cause unexpected pauses. After all, operating system knows more, what files are not likely to be touched, what files are changing often, etc.

seibelj 11y ago
And now we must rely on each OS to implement their own version of a virtual firmware, and do comparisons between different implementations, etc. etc. etc.
wtallis 11y ago
The industry is moving toward a mid-point of having the flash translation layer still implemented on the drive so that it can present a normal block device interface, but exposing enough details that the OS can have a better idea of whether garbage collection is urgently needed: http://anandtech.com/show/9720/ocz-announces-first-sata-host-managed-ssd-saber-1000-hms http://anandtech.com/show/9720/ocz-announces-first-sata-host... Moving the FTL entirely onto the CPU throws compatibility out the window; you can no longer access the drive from more than one operating system, and UEFI counts. You'll also need to frequently re-write the FTL to support new flash interfaces.
- nqzero 11y ago
  OS compatibility is important for laptops/desktops, but not in at least some database / server applications, and those are the applications that would benefit most from raw access
TD-Linux 11y ago
This has existed for a long time, see the Linux kernel's mtd infrastructure, and the filesystems designed to run on top of it (jffs, yaffs). It used to be used in a lot of embedded devices before eMMC became cheap, and is still used in things like home routers. I am not sure how well mtd's abstraction fits with modern Flash chips, though.
deleted 11y ago

japaw 11y ago

"2015 was the beginning of the end for SSDs in the data center." is quit a bold statement especial when not discussing any alternative. I do not see us going back to magnetic disk, and most new storage technology are some kind of ssd...

seabrookmx 11y ago
Yeah I thought that too. I think the author is specifically referring to 2.5" form factor SATA SSD's though.
scurvy 11y ago
My thoughts exactly. The article is quite inflammatory and tosses out some bold statements without really deep diving into them. My favorite: "Finally, the unpredictable latency of SSD-based arrays - often called all-flash arrays - is gaining mind share. The problem: if there are too many writes for an SSD to keep up with, reads have to wait for writes to complete - which can be many milliseconds. Reads taking as long as writes? That's not the performance customers think they are buying." This is completely false in a properly designed server system. Use the deadline scheduler with SSD's so that reads aren't starved from bulk I/O operations. This is fairly common knowledge. Also, if you're throwing too much I/O load at any storage system, things are going to slow down. This should not be a surprise. SSD's are sorta magical (Artur), but they're not pure magic. They can't fix everything. While Facebook started out with Fusion-io, they very quickly transitioned to their own home-designed and home-grown flash storage. I'd be wary of using any of their facts or findings and applying them to all flash storage. In short, these things could just be Facebook problems because they decided to go build their own. He also talks about the "unpredictability of all flash arrays" like the fault is 100% due to the flash. In my experience, it's usually the RAID/proprietary controller doing something unpredictable and wonky. Sometimes the drive and controller do something dumb in concert, but it's usually the controller. EDIT: It was 2-3 years ago that flash controller designers started to focus on uniform latency and performance rather than concentrating on peak performance. You can see this in the maturation of I/O latency graphs from the various Anandtech reviews.
- fleitz 11y ago

acquacow 11y ago

...unless you are Fusion-io, in which case, most of these problems don't affect you.

ddorian43 11y ago
why? isn't fusion-io based on ssd ?
- jjtheblunt 11y ago
  fusion-io (i believe, but please verify online) uses a spinning drive for frequent writes and an ssd for frequent reads, with software deciding what goes where, lessening the write traffic to the ssd, and thus wear to it.
  - acquacow 11y ago
    No, Fusion-io has nothing to do with spinning drives. They make PCI-e flash drives with an FPGA as the "controller" for the flash. There are multiple communication channels, so you can simultaneously read and write from the drives, and there are various tunings available to control how garbage collection works. They are the only "SSD" that doesn't hide all the flash behind any kind of legacy disk controller or group protocol like NVMe
  - jaysoncena 11y ago
    I think what you're trying to describe is apple's fusion drive.
- acquacow 11y ago
  They use flash like all other SSDs, but don't use a disk controller or any community defined protocols for mass compatibility. They use their own software and an FPGA to control the flash.
- pkaye 11y ago
  They have a special OS driver which moves the FTL closer to the OS. So one of the things they need is multi GB of memory from the OS to keep the FTL mapping tables. Also sudden power loss requires OS intervention to piece back the FTL structure (I seem to recall the original product taking 5 minutes to recover.) This also means you couldn't boot from an Fusion IO drive. I'm not sure if they fixed these issues on a more recent drive.

Hoff 11y ago

FWIW, the top-end HPE SSD models are rated for up to 25 writes of the entire SSD drive, per day, for five years. The entry-level SSDs are rated for ~two whole-drive writes per week. Wear gage, et al. http://www8.hp.com/h20195/v2/GetPDF%2Easpx%2F4AA4%2D7186ENW%2Epdf http://www8.hp.com/h20195/v2/GetPDF%2Easpx%2F4AA4%2D7186ENW%...

tim333 11y ago
Also maybe of interest, the Techreport The SSD Endurance Experiment. Their assorted drives lasted about 2,000 - 10,000 whole disk writes. http://techreport.com/review/27909/the-ssd-endurance-experiment-theyre-all-dead http://techreport.com/review/27909/the-ssd-endurance-experim...

velox_io 11y ago

The most exciting recent development in SSDs (until 3DXpoint is released), is bypassing the SATA interface, connecting drives straight into the PCIe bus (no more expensive raid controllers). Just a shame hardly any server motherboards come with M.2 slots right now. The 4x speed increase and lower CPU overhead means it is now possible to move RAM only applications (for instance in-memory databases) to SSDs, keeping only the indexes in memory. Yeah, we've been going that way for a while, just seems we've come a long way from expensive Sun e6500's I was working with in just over a decade ago.

wtallis 11y ago
M.2 slots don't make much sense for servers, at least if you're trying to take advantage of the performance benefits possible with a PCIe interface. Current M.2 drives aren't even close to saturating the PCIe 3.0 x4 link but they're severely thermally limited for sustained use and they're restricted in capacity due to lack of PCB area. Server SSDs should stick with the traditional half-height add-in card form factor with nice big heatsinks.
- scurvy 11y ago
  2.5" NVMe is fine and more scalable than add-in cards (more slots) with increased serviceability. The equivalent add-in card is usually much more expensive than the 2.5" version.
- lsc 11y ago
  most of the NVMe backplanes I've seen give full 'enterprise 2.5" drive clearance to the thing, so if they are actually as thick as consumer SSDs, as most current SATA 'enterprise SSD' are, there's plenty of room for a heatsink without expanding the slot. The supermicro chassis (and I've only explored the NVMe backplanes from supermicro) usually put a lot of effort into drawing air through the drives, so assuming you put in blanks and stuff, the airflow should be there, if the SSD are setup to take advantage of it.
  - wtallis 11y ago

bravura 11y ago

I need an external disk for my laptop that I leave plugged in all the time. What is the most reliable external hard drive type? I thought SSDs were more reliable than spinning disks, especially to leave plugged in constantly, but now I'm not as sure.

Thorbears 11y ago
This article, and the majority of the comments here, are about using SSDs in server environments, where permanently high load and zero downtime is the norm. And it doesn't even seem to be about SSDs vs HDDs, it is about SSDs vs future technologies. For personal use, SSDs outperform HDDs in just about every aspect, if you can afford the cost, an SSD is the better choice. And there is nothing mentioned here about downsides of leaving a drive plugged in and powered on at all times.
ScottBurson 11y ago
I still don't trust SSDs as much as I do spinning disks. While neither kind of drive should be trusted with the only copy of important data, I would say that drives used for backup, or for access to large amounts of data that can be recovered or recreated if lost and where the performance requirements do not demand an SSD, might as well be HDDs -- they're cheaper, and arguably still more reliable. If the workload is write-heavy, HDDs are definitely preferred as they will last much longer. While all disks can fail, HDDs are less likely to fail completely; usually they just start to develop bad sectors, so you may still be able to recover much of their contents. When an SSD goes, it generally goes completely (at least, so I've read). So it depends on your needs and how you plan to use the drive. For light use, it probably doesn't matter much either way. For important data, you need to keep it backed up in either case. SSDs use less power, particularly when idle, so if you're running on battery a lot, that would be a consideration as well.
- cnvogel 11y ago
  Anecdotal evidence: 2.5" SATA HDD failed for me suddenly just last Tuesday, SMART was fine before, both the attributes and a long (complete surface scan) selftest I did a few weeks ago after I got this lightly used notebook from a colleague (I only needed for tests). I think what people experience is that the sudden death of SSDs doesn't occur more often than on HDDs. But with the mechanical issues and slowly building up of bad sectors gone, sudden death is probably the only visible issue left. (Just my personal opinion.)

ak217 11y ago

This article has a lot of good information, but its weirdly sensationalistic tone detracts from it. I appreciate learning more about 3D Xpoint and Nantero, but SSDs are not a "transitional technology" in any real sense of the word, and they won't be displaced by anything in 2016, if nothing else because it takes multiple years from volume capability to stand up a product pipeline on a new memory technology, and more years to convince the enterprise market to start deploying it. The most solid point the article makes is that the workload-specific performance of SSD-based storage is still being explored, and we need better tools for it.

nostrademons 11y ago
I got the sense that it was a PR hit for Nantero, bought and paid for. Notice the arc of the article: it says "[Popular hot technology] is dead. [Big vendors] have recently introduced [exciting new product], but [here are problems and doubts about those]. There's also [small startup you've never heard of] which has [alternative product] featuring [this list of features straight from their landing page]." Usually these types of articles are designed to lead people directly to the product that's paying for the article. Sensationalistic is good for this; it gets people to click, it gets people to disagree, and then the controversy spreads the article across the net. Seems like it's working, in this case.
- Laforet 11y ago
  Robin Harris has been advocating for the abolishment of block abstraction layer for a couple of years now and this piece is consistent with his usual rhetoric
- Dylan16807 11y ago
  Eh, it listed it as "promises" and talked the same way about Adesto. It's reasonable to say "this is the basic claim of the product; we'll see if they get there" without it being PR.

fleitz 11y ago

Given the number of IOPS SSDs produce they are a win even if you have to chuck them every 6 months.

ilaksh 11y ago

Article is garbage. Basically "I told you so" by someone who never got up-to-date after the first SSDs came out and found some numbers to cherry pick that seemed to support his false beliefs.

rodionos 11y ago

>log-structured I/O management built into SSDs is seriously sub-optimal for databases and apps that use log-structured I/O as well This assert piqued my interest given that my hands-on experience with HBase speaks to the contrary. The paper by SanDisk they refer to https://www.usenix.org/system/files/conference/inflow14/inflow14-yang.pdf https://www.usenix.org/system/files/conference/inflow14/infl... seems to suggest that most of the issues are related to sub-optimal degragmentation by the disk driver itself. More specifically, the fact that some of the defragmentation is unnecessary. Hardly a reason to blame the databases and can be addressed down the road. After all, GC in Java is still an evolving subject.

otakucode 11y ago

Can't wait for the inevitable discovery that NAND chips are being price-fixed. You would think after the exact same companies price-fixed RAM and LCD panels that peoples radars would go off faster. You expect me to believe that the metals and neodymium magnets and now helium containing drives are cheaper to manufacture than an array of identical NAND chips? NAND chips which are present in a significant percentage of all electronic products purchased by anyone anywhere? When a component is that widely used, it becomes commoditized. Which means its price to manufacture drops exponentially, not linearly like the price drops of NAND has done. This happened with RAM and LCDs as well. When you can look around and see a technology being utilized in dozens of products within your eyesight no matter where you are, and those products are still more expensive than the traditional technology they replace, price-fixing is afoot. I am open to being wrong on this, but I don't think I am. Can anyone give a plausible explanation why 4TB of NAND storage should cost more to manufacture than a 4TB mechanical hard drive does, given the materials, widespread demand for the component, etc?

pjc50 11y ago
"Apples do not cost the same as oranges, therefore oranges are being price-fixed" is not a convincing line of reasoning. The two technologies are very different and NAND storage is much newer, and has always been much more expensive than disk storage. The correct thing to compare NAND prices to is other chips that are being fabbed at the same process node, by die area.

unixhero 11y ago

Great thread.