21 ms·
SSD Storage: 2018 in review
- camerondoll 8y agoI wish Intel Optane nvme sticks had more capacity. The 120gb drive turned my i7 6500u laptop TWICE as fast as desktop AMD 1700x with regular drive. For example Python tests on a fairly big app ran for 3 minutes on a laptop and same tests about 7 minutes on a supposedly more powerful desktop machine. Edit: by regular I mean 960 evo
- seanalltogether 8y agoNot sure if I'm asking this correctly but, all SSDs now have their own internal filesystem for managing access to blocks right? Are we getting to a point where traditional filesystems like NTFS or ext4 will simply go away? Or will they still stick around and just act as lightweight layers on top of the SSD filesystem?
- iwalton3 8y agoThose filesystems seem to handle different design requirements though. I also don't trust most vendors to properly encrypt the drives in the firmware [1], so I end up putting dm-crypt over most drives. It also doesn't account for raid setups, which are usually done at the block level. (Although I like snapraid for some use cases.) [1] https://www.welivesecurity.com/2018/11/15/security-researchers-bypass-encryption-self-encrypting-drives/ https://www.welivesecurity.com/2018/11/15/security-researche...
- megous 8y agoI think it's unlikely that manufacturers will want to turn their consumer SSD devices into a dumb flash storage and move their FTL into the open (like a API layer inside the Linux kernel). It's where their added value lies.
- Dylan16807 8y agoProbably not putting drives in charge of the entire filesystem, but object storage would be a good way to let the drive manage itself and take care of the details of where to store everything. The SCSI commands for object storage have existed for a decade but so far haven't seen much uptake.
- Moral_ 8y agoThose flash translation layers on the SSDs are generally used to maintain QoS/and do wear leveling, block management etc. I don't think we'll see file systems go away, but what we may see is more knowledge pushed into the file system, instead of keeping it down in the controller. We have started to see that with the advent of LightNVM which exposes a more RAW API into the drive and the FTL is maintained in the kernel. The current "generic" implementation of the LightNVM FTL is called pblk: https://elixir.bootlin.com/linux/latest/source/drivers/lightnvm/pblk-core.c https://elixir.bootlin.com/linux/latest/source/drivers/light...
- wtallis 8y agoThe flash translation layers in SSDs implement things equivalent to the journalling functionality in modern filesystems, but they don't really provide any hierarchical organization of data. So you'll still need a normal host-based filesystem for the foreseeable future, but filesystem design can be influenced by the assumption that it's running atop flash storage. There are several efforts underway to provide more specialized software interfaces to SSDs. Several vendors have produced models that expose a key-value store instead of fixed-size block storage; with those drives, you can throw away RocksDB and speak directly to the drive with the same semantics (subject to limitations on supported key and value sizes). There are a few competing standards for open-channel SSDs that move some of the FTL onto the host CPU so that the journalling overhead doesn't have to exist at multiple layers; the different solutions here vary in terms of how much complexity they move to the CPU vs. how much abstraction the SSD still provides for the sake of software portability. Most of the potential benefits this approach provides are being subsumed by extensions to the NVMe protocol that allow the host and SSD to exchange optional hints about data layouts, GC status, etc. SK Hynix recently announced they're working on a SSD with transactional storage support, so that the host can send multiple write commands and either commit or abort the transaction as a whole.
- kbuck 8y agoAt the SSD level, the drives are actually doing block management, not file management. You still need a filesystem to store metadata, manage the layout of the data, etc. An example of something that a SSD's controller does that the operating system/filesystem doesn't have to worry about is managing bad blocks. If the SSD detects a bad block, it will replace it with a working block and update the data used by its flash translation layer to move the blocks around. This is completely opaque to the operating system; as far as it knows its underlying storage works exactly the same (until there are so many bad blocks that the drive can't keep up this convenient deception). An example of something a filesystem does that the SSD doesn't provide is storing operating system-specific file metadata, such as permissions, creation times, multiple data streams, directory layouts, etc. SSDs deal only in blocks of data, not arbitrarily-sized units, nor metadata. The reason that this behavior isn't more tightly-integrated is because some of the details of managing the underlying flash blocks tend to be specific to type of flash, or even different models of flash. For example, the article mentions QLC flash becoming mainstream - we're finally getting to this point because previously, QLC was so difficult to manage that your filesystem had to be aware that it was writing to QLC flash to use it effectively. There are a few filesystems designed for direct flash management like yaffs[0], but this isn't quite as efficient as a SSD's dedicated processor and software stack. [0]: https://yaffs.net/ https://yaffs.net/
- amelius 8y ago> This is completely opaque to the operating system; as far as it knows its underlying storage works exactly the same (until there are so many bad blocks that the drive can't keep up this convenient deception). Is there a way for the drive to tell the fs that a block is bad? Or does the drive simply keep a bunch of blocks apart just in case?
- wtallis 8y agoHard drives and SSDs both keep a pool of spare blocks so that they can remain functional after having to retire some defective blocks from use. This spare pool tends to be much larger for SSDs. Ordinary IO doesn't convey anything about whether a block had to be retired in the process, but there are SMART indicators that track this stuff.
- alxmdev 8y agoExciting times indeed, just type "ssd" in your favorite search box. I'm about to upgrade to a 512GB SSD that costs half what an SSD half its size used to cost 5 years ago! Starting this year, I'm considering getting a low-end 120GB for $25 every year on January 1st to use as a write-once backup for all my work.
- sigstoat 8y agoedit: i cannot arithmetic today, or sanity check my arithmetic. sorry. sigh.
- fourthark 8y agodo you trust a startup still to exist in 41 years, and to honor your plan that long?
- sigstoat 8y ago(my arithmetic was wrong, so 41 years was garbage, but addressing the more general point: ) the corporate entity that is currently doing business as backblaze doesn't need to last 41 years, or even 41 more days. the stored data is all a valuable stream of revenue from customers who'd like it preserved, so unless their whole model is unsustainable, i'd expect the company to be sold to/absorbed by another business. as for honoring the plan, i don't think cloud storage rates are going to go _up_. so maybe they wouldn't want me as a customer at some point, but so long as they're taking consumer business i can't imagine the pennies per GB*month going up unless there is some sort of massive societal upheaval.
- coder543 8y agoI'm not sure how you got that $25 = 41 years of 120GB of storage on Backblaze. Their current pricing appears to be $0.005/GB-month... which would be $295.20 for 41 years of 120GB of storage. > i wouldn't trust a cheap SSD to still work in 41 years I wouldn't trust almost any company to still exist in 41 years, including Backblaze. Preserving data over that length of time will probably require active management of the data in some form or another. DVD or Blu-ray discs might last that long... but it's hard to trust any storage medium to last for 41 years. Modern storage technology just hasn't existed that long.
- wtallis 8y agoTwo interesting updates based on what's been announced so far in 2019: PCIe 4.0 will be arriving in the consumer market this year, with a new generation of AMD Ryzen CPUs providing host support, and at least one or two consumer-class NVMe SSD controllers supporting PCIe 4.0 should be ready to start shipping in retail products by the end of the year. (The enterprise/datacenter storage market's transition is already well underway.) Seagate became the first vendor I'm aware of to start marketing a SSD to the prosumer/SMB market for NAS usage. It's a rebadge of one of their recent enterprise SATA drives and isn't even using QLC NAND so it's probably going to be pretty pricey, but the idea of a solid-state NAS is no longer completely laughable.
- devereaux 8y agosolid state NAS not not laughable, just expansive. RAID 10 a few NVME and you can get decent throughput (and storage size) with existing technology.
- da_chicken 8y agoI don't know that I'd consider RAID 10 for prosumer. RAID 1, absolutely. RAID 10 seems like needlessly adding more complexity and higher failure rates.
- Spooky23 8y agoGreat point. Complexity is killer in home setups.
- Dylan16807 8y agoHaving to sort files among multiple volumes is also complexity and can lead to mistakes.
- leetcrew 8y ago> RAID 10 a few NVME and you can get decent throughput (and storage size) with existing technology. is there a good reason to do this in a consumer setup? max realistic throughput over gigabit ethernet is only ~120MB/s, which can easily be saturated by sequential reads or writes to/from a single modern spinning-rust drive.
- franciscop 8y agoI've been following the news and it's amazing how SSDs are still progressing, a bit slower but similar to Moore's Law. Prices for Samsung SSD 1TB external USB products: - 2015 $600: Samsung T1 http://www.thessdreview.com/our-reviews/samsung-portable-ssd-t1-review-1tb-price-speed-capacity-security/ http://www.thessdreview.com/our-reviews/samsung-portable-ssd... - 2016 $430: Samsung T3 https://www.storagereview.com/samsung_portable_ssd_t3_review https://www.storagereview.com/samsung_portable_ssd_t3_review - 2018 $200: Samsung T5 https://www.androidheadlines.com/2018/12/samsung-t5-portable-ssd-1tb-198-amazon-holiday-deals-2018.html https://www.androidheadlines.com/2018/12/samsung-t5-portable... Another incredible exciting point is the amount of competition coming up, it's not just 2-3 large vendors anymore. If things continue at this pace it seems sensible to expect a 10TB SSD consumer drive for under $1000 within a couple of years. Note: not afiliated at all, just a happy consumer.
- baroffoos 8y agoI await the day I can fill my NAS with SSDs. Might save on power usage as well.
- elorant 8y agoNot sure if it's a good idea though. SSDs tend to fail without warning. I guess that's why we have RAID but still I wouldn't want my backup storage to be so volatile.
- ggm 8y agoYou think HDD can't fail without warning? OK, I get that SSD don't implement S.M.A.R.T but..
- segfaultbuserr 8y ago> I get that SSD don't implement S.M.A.R.T Despite the S.M.A.R.T data is less predictive on SSDs, they do have it, at least for SATA disks...
- nimbius 8y agosurprised these are still a thing. I bit the bullet and moved to M.2 storage across pci lanes. It makes my current SSD's feel like spinning disk. https://en.wikipedia.org/wiki/M.2 https://en.wikipedia.org/wiki/M.2
- aidenn0 8y agoTFA includes discussions of all SSDs, M.2 formfactor included.
- reasonablemann 8y agoWhy is there competition in the SSD market but not in the RAM market?
- wtallis 8y agoI'm not sure. The pattern in the DRAM market seems to have been that only the biggest, most successful manufacturers can weather price crashes while still investing enough in R&D to stay on the leading edge. In the NAND flash market, we may simply not have yet seen any oversupply serious enough to do that, for all that we're in the middle of an oversupply-driven price crash. It is probably the case that it's easier to stay in business with inferior NAND; Samsung beat everyone else to 3D NAND by a few years but Toshiba still made a killing off cheaper, lower quality planar NAND, and Intel/Micron didn't seem to suffer meaningfully from their first-generation 3D NAND being so slow. Now everyone other than SK Hynix has caught up to Samsung and the number of players in the market is actually increasing.
- ATsch 8y agoThe recent scandals about DDR4 price fixing lead me to believe it's probably collusion.
- kingosticks 8y agoCould it also be partly due to the infancy of the technology and the market? You don't see new RAM vendors and there are fewer avenues of improvement left to explore. That leaves a race to the bottom which is pretty unexciting for everyone.
- jaytaylor 8y agoWhoa, check out that Samsung NF1 form-factor [0]. Looks like you could pack a ton of small finger-sized SSD drives vertically into a small space. [0] https://images.anandtech.com/doci/13752/IMGP2727_575px.jpg https://images.anandtech.com/doci/13752/IMGP2727_575px.jpg This is one way in which it's a cool time to be alive. I'll never forget circa 1996 when I waited 20+ minutes for my Apple Performa 550 to read and load a 30MB file from the scsi disk into memory. All so quicktime could play a grainy, low-resolution video clip for all of 30 seconds. At the time I was like "That's almost 1 minute per second of video! WTF? That sucks."
- deleted 8y ago[deleted]
- wtallis 8y agoThat picture is actually the EDSFF 1U Short form factor, which is the closest competitor to Samsung's NF1 out of the variety of form factors defined by EDSFF. Both allow for about 36 drives in the front of a 1U system.
- woolvalley 8y agoThat must get really hot, I wonder how well it cools?
- wtallis 8y agoThere's no backplane to get in the way of airflow, so it is quite a bit easier to cool than a typical hot-swap bay full of 2.5" drives: https://images.anandtech.com/doci/13218/IMGP2722.jpg https://images.anandtech.com/doci/13218/IMGP2722.jpg The new connectors between the drives and the mid-plane board are probably the most important innovation these new form factors bring to the table, though something similar could also be done for existing SAS/U.2 connectors.
- manmal 8y agoDon't they just use 1W each, making up 16W in total? I might be dead wrong. But 16W should be easily transported off by the metallic case and a smallish fan?
- rb808 8y agoThe other amazing thing is size. How much smaller and cooler m2 drives are compared to old 3.5" HDs. If you have integrated graphics, PCs can now be tiny. The big tower cases are really for gamers only.
- zanny 8y agoEven gamers don't need towers, games don't benefit much from anything > 4 cores and the only limiting factor so all you really need is a mini-itx motherboard, 8-16GB of ram, and a beefy GPU attached. There are plenty of console sized cases out there you can build in that can fit a 2080 or Vega card. Big cases are really for tons of mechanical drive capacity, > 8 core CPUs, addin cards like network switches, or a ton of GPUs in a compute cluster.
- kevin335200 8y agoJust bought MX500 2TB in black friday for about 200 euros. Maybe it will getting cheaper this year, but 0.1 EUR/GB is already a good deal for me
- deleted 8y ago[deleted]