14 ms·

Everything I Know About SSDs

352 points by classified 7y ago

notacoward 7y ago
The part about filesystems is slightly incorrect. > The way the file system handles this is incompatible with the workings of NAND flash. That's true of most conventional filesystems, but log-structured filesystems are much more flash-friendly. That's why there has been a resurgence of interest in them, and also why a typical flash translation layer bears a striking resemblance to a log-structured FS. There are also flash-specific filesystems. > to an HDD all sectors are the same. This is not true because of bad blocks. Every disk has a reserve of blocks that can be remapped in place of a detected bad block, transparently, much like flash blocks are remapped. Beyond that, it's also useful for a disk to know which blocks are not in use so it can return all zeroes without actually hitting the media. There are special flags to force media access and commands to physically zero a block, for the cases where those are needed, but often they're not. Trim/discard actually gets pretty complicated, especially when things like RAID and virtual block devices are involved.
- rckoepke 7y ago
  > to an HDD all sectors are the same. Also I believe some humans, and (filesystems?) intentionally stored certain data towards the inside/outside of the HDD because the simple cylinder geometry allowed faster reads in those regions. However, I'm not seeing conclusive proof that modern HDDs show performance variation with respect to radius.
  - monocasa 7y ago
    Io schedulers would also understand the geometry and reorder commands to minimize seek distance (and therefore seek time).
    - notacoward 7y ago
      I used to do some of that a long time ago, and it was actually kind of fun. There's a whole different set of optimizations to do on flash, both for performance and (more importantly in many use cases) power efficiency, and I think that would be fun too.
    - tzs 7y ago
      At some point, I think around the mid '90s or early 2000s but maybe earlier, though, seek time was fast enough on widely available drives that on average for random access you spent about as much time on the right cylinder waiting for the sector you wanted to rotate to under the head as you did on seeking to the right cylinder. You could get some decent gains then if you made your scheduler take rotation into account. A long seek that arrived just before the target sector came under the head could be faster than a short seek that would arrive just after the sector passed the head. On the other hand, taking rotation into account could make the scheduler quite a bit more complex. You needed a model that could predict seek time well, and you needed to know the angular position of each sector in its cylinder. I don't think that there were any drives that would tell you this. SCSI drives wouldn't even tell you the geometry. IDE drives would tell you a geometry, but it didn't necessarily have anything to do with the actual geometry of the drive. At the time I worked at a company that was working on disk performance enhancement software (e.g., drivers with better scheduling, utilities that would log disk accesses and then rearrange data on the disk so that the I/O patterns in the logs would be faster [1], and that sort of thing). We had a program that could get the real disk geometry. It did so by doing a lot of random I/O and looking at the timing of when the results came back. If there were no disk cache, this would be fairly easy. (Well, it didn't necessarily get the real geometry, but rather a purported geometry and seek and rotational characteristics that could predict I/O time well). For instance, read some random sector T, then read another random sector, then read T again. Look at the time difference between when you started getting data back on the two reads of T. This should be a multiple of the rotation time. If the disk has caching that can still work but you need to read a lot of random sectors between the two reads of T to try to get the first read out of the cache. Anyway, we had to give up on that approach because the program to analyze the disk took a few days of constant I/O to finish. Management decided that most consumers would not put up with such a long setup procedure. [1] Yes, that could mean that it would purposefully make files more fragmented. A fairly common pattern was for a program to open a bunch of data files and read a header from each. E.g., some big GUI programs would do that for a large number of font files. Arranging that program and those data files on disk so that you have the program code that gets loaded before the header reads, then the headers of all the files, and then the rest of the file data, could give you a nice speed boost. The flaw in this method is that, to use the above example, if another big GUI program also uses those same font files, the layout that makes the first program go fast might suck for the second program. If you've got a computer that you mostly only use for one task, though, it can be a viable approach.
  - wtallis 7y ago
    > However, I'm not seeing conclusive proof that modern HDDs show performance variation with respect to radius. They definitely still do; it's fundamental to drives that run at fixed RPM but maintain high areal density across the entire platter. One of the 1TB drives I have lying around does about 183MB/s at the beginning of the disk, 148MB/s in the middle, and 97MB/s at the end.
    - ascar 7y ago
      I'm really interested in how you benchmarked that? Is there a "simple" way to specify the physical location on a HD data should be written to?
      - wtallis 7y ago
        Hard drives generally use a fairly simple linear mapping between LBAs and physical location. Low LBAs are on the outer edge of the platter where transfer speeds are highest, and high LBAs are on the inner edge where speeds are lowest. Unlike SSDs, hard drives don't need wear leveling, so there's no reason to break from that pattern except in the relatively rare instance of damaged sectors. At smaller scales, the layout of individual tracks can vary quite a bit, but that doesn't have as much impact on overall sequential transfer speed. See http://blog.stuffedcow.net/2019/09/hard-disk-geometry-microbenchmarking/ http://blog.stuffedcow.net/2019/09/hard-disk-geometry-microb...
        zozbot234 7y ago
        Your link was discussed @ https://news.ycombinator.com/item?id=21492345 https://news.ycombinator.com/item?id=21492345
        wtallis 7y ago
        Yep. I found it interesting enough to bookmark, and now I tend to pull it up when somebody wants to know how hard drives really work.
        mehrdadn 7y ago
  - kabdib 7y ago
    This is still very much true, to the point that game consoles will reserve areas of rotating media for different purposes, depending on speed (e.g., disk caches on the Xbox are on the outer tracks). For optical media the layout of assets can be critical to the user experience. You probably want your startup assets located on the outer, faster tracks.
  - marcosdumay 7y ago
    People reduce the number of cylinders to keep seek time under control. It just happens that if you are going to choose what cylinders to use, you better choose the inner ones that are slightly faster.
    - Hackbraten 7y ago
      Wouldn’t the outer ones be slightly faster in terms of both transfer rate and head travel time?
  - CrazyStat 7y ago
    I remember reading a blog post ca. 2005, before SSDs were common, in which the author explained how he had identified all the files his system needed to boot and the order in which they were read from disk, and then contrived to put them all in the correct order in a contiguous block at the start of the drive to eliminate seeks during boot entirely. This apparently shaved a couple seconds off his boot time.
    - dspillett 7y ago
      There are automated solutions to this. Windows has had similar tricks built in (at various levels of cleverness, depending on the Windows variant you run) for a while now: https://en.wikipedia.org/wiki/Prefetcher https://en.wikipedia.org/wiki/Prefetcher No doubt there are similar options for Linux and other OSs. A similar trick I've seen is using a small SSD as a manual cache: copy in the files that need to be fast and mount it with the larger filesystem (on slower drives) as a unioned filesystem. Though just using block device based automatic cacheing may be easier and safer than rolling your own mess. There are a few options for Linux though some are not currently maintained (see https://serverfault.com/questions/969302/linux-ssd-as-hdd-cache https://serverfault.com/questions/969302/linux-ssd-as-hdd-ca... amongst other places), and some IO controllers support it directly (even some motherboards have this built in) without needing to bother your OS with the details at all.
- guenthert 7y ago
  > but log-structured filesystems are much more flash-friendly. That's why there has been a resurgence of interest in them I could swear I read such in 2010 -- did that actually happen in the last ten years?
  - osmarks 7y ago
    f2fs (https://en.wikipedia.org/wiki/F2FS https://en.wikipedia.org/wiki/F2FS) is now popular for Android-based mobile devices.
    - guenthert 7y ago
      That counts, thanks!
- rodgerd 7y ago
  He's also inaccurate in characterizing all cells as "1 or 0"; digital electronics have always been a cut-off of an underlying analogue value (such as voltage).
  - outworlder 7y ago
    They explain this in more detail later on. See "SSD Reads" and "interpreting the results".
- namibj 7y ago
  I briefly searched for a source of (mostly) FTL-less microSD cards (I'd be fine with them doing wear leveling-related remapping of whole blocks, as long as it's power-loss-safe, but I don't want them to expend any effort on emulating 512/4k random write capability). I didn't find anything but bunnie's blog entry[0] on hacking/re-flashing the firmware on the card's controller. Due to how important it is to have a good FTL, and the pretty much native suitability of log-structured data formats, like those based on RocksDB or InfluxDB (both not unlikely for data handled and stored on RPi-like SBCs), it'd be much better for reliability and performance reasons to let these LSM engines deal with NAND flash's block erase/sector write/sector read behavior. [0]: https://www.bunniestudios.com/blog/?p=3554 https://www.bunniestudios.com/blog/?p=3554
Ariez 7y ago
I was under the impression that if you do not encrypt an SSD from the first use, then any attempt at overwriting with 0s is futile, as well as any other method to securely delete the files. The files will be easily recovered. This guy seems to say the opposite, in that the files are "simply not there anymore", contrary to everything I've read: who's right here?
- blattimwind 7y ago
  In practice all SSDs are always encrypted because they use the encryption to whiten the data written to them. That's why "Secure Erase" takes less than a second on SSDs, it doesn't erase anything but the key.
  - e12e 7y ago
    But AFAIK this isn't true for all solid state storage, like cheap USB memory keys. These would generally also benefit from log structured filesystems - but really should be encrypted too. (which is one great promise of zfs on linux/openzfs - cross-platform encrypted removable storage).
  - nemo1618 7y ago
    Interesting! Do you happen to know which encryption algorithm is used? I would think that, if the only goal is whitening (as opposed to robust security), a fairly weak algorithm would be used, or perhaps a strong algorithm with a reduced number of rounds.
    - theandrewbailey 7y ago
      The algorithm probably depends on the drive, but AES-256 is common due to hardware acceleration. Read a SSD spec sheet sometime, it will likely mention it, along with supporting TCG Opal (the self encrypting drive standard).
    - sigstoat 7y ago
      the hardware is going to use AES because their ASIC vendor will have well tested AES IP that they can just throw down on the chip. any other algorithm would require massive development effort for zero benefit. and by using AES they can probably claim to satisfy some security standards that will make their marketing people happier.
rahuldottech 7y ago
There's some interesting content about SSDs and HDDs here: SSDs: https://superuser.com/questions/tagged/ssd?tab=Votes https://superuser.com/questions/tagged/ssd?tab=Votes HDDs: https://superuser.com/questions/tagged/hard-drive?tab=Votes https://superuser.com/questions/tagged/hard-drive?tab=Votes
cattlemansgold 7y ago
There are a few technologies that I’ve tried very earnestly to understand, only to find out that it’s basically black magic and there’s no use in trying to understand it. Those things are modern car transmissions, nuclear reactors, and SSDs.
- 24gttghh 7y ago
  A very basic nuclear reactor can be explained pretty simply I think. You enrich a bunch of let's say uranium. Pack it together in a rod, and put a bunch of those rods in a pond. Those rods have controlled (ideally) nuclear decay from their being in close proximity to other rods which generates a lot of heat, which is transferred to a separate cooling loop that boils water to make steam which drives an electric turbine. Now I'm no nuclear scientist so please be forgiving with that description, but that's how I understand them to work :) I can't even begin to explain how an SSD works, but I know there are no moving parts besides electrons. edit: moved the "(ideally)"
  - zzzcpan 7y ago
    You can explain SSD as a controller you send commands to that writes data to and reads from a log structured storage on top of raw flash.
  - cattlemansgold 7y ago
    I guess what I meant is that the overall concepts are understandable. Nuclear fuel gets hot, boils water, drives a turbine. For transmissions, different sized gears allow things to turn at different rates. But as soon as I dive into the details, I get lost. How exactly can you control the nuclear decay? How exactly does do the gears in the transmission move around and combine with eachother to create a specific gear ratio? These concepts probably are probably pretty simple for a lot of people, but they just make my head spin.
    - toast0 7y ago
      > How exactly can you control the nuclear decay? That's what the control rods are for. The uranium in one fuel rod in isolation decays at whatever natural rate, which would warm water but not boil it, and placing the rods near each other allows for the decay products (high energy particles) to interact with other fuel rods and induce more rapid decay. The control rods slot in between the fuel rods, and absorb the decay products without inducing further nuclear decay. Usually these are graphite rods. > How exactly does do the gears in the transmission move around and combine with eachother to create a specific gear ratio? It really depends on the specific transmission, a manual transmission is using the shift lever movement to move the gears into place. An automatic transmission most likely uses solenoids to move things (a solenoid is basically a coil of wires around a tube with a moveable metal rod inside, when you put current through the wire, the metal rod is pulled into the tube, you attach the larger thing you want to move to the end of the rod (sometimes with a pivot or what not), and use a spring, another solenoid, or gravity, etc to make the reverse movement. A solenoid by itself gives you linear movement, if you need rotational movement, one way to do that is have a pivot on the end of the solenoid rod, then a rod from there to one end of a clamp on a shaft, then when the solenoid pulls in its rod, the shaft will rotate (this is the basic mechanism for pinball flippers).
fareesh 7y ago
I wonder if anyone here has experienced something similar: I have a Samsung Evo 860 SSD. Sometimes after powering on my desktop the BIOS either "forgets" the drive (sets some other drive as primary boot drive) or doesn't recognize it at all. The non recognition issue goes away after I power off and power on. It's been this way for about 8 months. Happens every 1/15 times I power on. I've heard it may have something to do with "sleep mode" or something like that. I always shutdown via software though.
- cptskippy 7y ago
  That sounds like the drive is failing to initialize in time. Have you tried enabling a POST test or Boot Delay? I suspect the problem might magically go away if you do.
  - fareesh 7y ago
    I'll try it out
- leetcrew 7y ago
  might be an innocent config issue, but this usually means the drive detection timed out during boot, which can be an early sign of drive failure. one of my old ssds behaved this way for a while before never being detected again. make sure you have anything important backed up.
- toast0 7y ago
  I had an ssd/system combination that would not communicate at sata 3, and either the bios wouldn't try at sata 2 or it would timeout detection too soon after. If your bios lets you set a sata revision, it's worth a try (mine didn't, but I used a sketchy download of the AMI Bios Editor to adjust the default; I couldn't figure out how to use the editor enough to make the setting visible). And also maybe reseat the cables.
- cmurf 7y ago
  Is the drive's firmware current? And what about the logic board firmware?
cptskippy 7y ago
The CSS of this site manages to making reading this article equality inconvenient and painful on both Mobile and Desktop.
- _hao 7y ago
  Same experience for me. Text seems readable and can be easily followed at 250% zoom (at least for me)
- amarshall 7y ago
  Indeed. Thankfully, Reader View in both Firefox and mobile Safari remedies it.
creeble 7y ago
Anyone know how TRIM works with Linux? I find myself copying entire partitions between SSDs from time to time, is there a utility to clear the destination SSD before copy? Is it possible to do the same for an SD card, so that writing a new Raspi OS to it doesn't do unnecessary garbage collection?
- leetcrew 7y ago
  I believe all major OSes support trim in a reasonable way now. afaik it needs hardware support though, so idk how it would work for an SD card.
  - cm2187 7y ago
    Unless you have a RAID card in the middle.
    - leetcrew 7y ago
      I don't mean to say it wouldn't work, just that I don't know enough about sd cards to say whether it would/should.
      - cm2187 7y ago
        Sorry, I meant for TRIM being passed down to an SSD.
        leetcrew 7y ago
        oh gotcha. not sure why I thought it was reasonable to think you were talking about a bunch of SD cards in a raid array...
  - monocasa 7y ago
    MMC defines trim as an optional command. But maybe to your point I've generally only seen it implemented on eMMC chips, not SD cards.
- 7y ago
cbdumas 7y ago
I love a minimal, text-only website as much as the next crotchety HN reader, but a little bit of CSS goes a very long way in terms of readability. Edit: Upon further inspection I see that this page was designed to be hard to read. Very curious.
- occamrazor 7y ago
  Reader view helps a lot, and with such a simple website it is guaranteed to give a good result.
  - dr_zoidberg 7y ago
    Not just a lot, it actually makes it readable. However I do feel this article needs a few graphs and diagrams. I've written [0] about data recovery and SSD/Flash storage is a massive beast to tackle even with diagrams to give you a vague idea of what's going on. [0] in old fashioned tree-based paper, unfortunately
    - ShroudedNight 7y ago
      Is there an available method to acquire some of these marked-up dead trees?
  - 867-5309 7y ago
    not if you're reading via Materialistic via Android browser with no responsive design or zoom ~,~
    - Flott 7y ago
      In Materialistic you can hit the red round button that appears bottom right of the screen. That way you can zoom. It won't help you much with this article, but it can be handy sometimes.
- computator 7y ago
  If the author is reading this, I’d like to ask him to please run his article through a spell checker. All the effort to write a nice article is spoiled by having dozens of typos and spelling mistakes.
ololobus 7y ago
"As for writes, at 1 gb a day - far more than my current rate of data use - it would take the same 114 years to reach 40 tb." Maybe it works for author's very specific system or use-case, but on my personal MBP laptop with very occasional usage pattern--some days I do not use it at all during the week--I end up with 10 GB per day of writes on average. That way it will be 11.4 years already, not so many. And I do not do something very disk-expensive on it like torrents downloading or database testing, only some general development task, watching online videos, web surfing, docs, etc.
- gowld 7y ago
  How much RAM do you have? And how many Chrome tabs?
  - ololobus 7y ago
    16 GB of RAM and I use Safari mostly and Firefox from time to time with less than 20 tabs usually (less than 10 in average, I guess). I check Data written in the Activity monitor — Disk.
- ls612 7y ago
  I have an NVME SSD as a boot drive in my desktop and three years and change in (say 1200 days) it’s used 40tb of write (out of an advertised 400tb endurance so I’m not too worried). That works out to about 30gb of writes per day, which seems about right for medium to heavy use. I guess what I’m saying is that for modern SSDs I don’t think write endurance is a binding constraint in most cases.
  - wtallis 7y ago
    For client/consumer SSDs, most vendors seem to view 50GB/day as plenty of write endurance for mainstream, non-enthusiast products. Virtually all retail consumer SSDs have a warranty that covers at least that much, and usually several times more for larger drive capacities (since write endurance is more often specified in drive writes per day).
computator 7y ago
> Deleted file recovery on a modern SSD is next to impossible for the end user OK, he’s talking about SSDs, but I want to mention that I’ve easily recovered many large deleted files from SD or micro-SD cards (formatted as FAT32 or exFAT) using Norton Unerase or an equivalent utility. Are the controllers for SSDs that different from the controllers for SD cards? Has anyone tried Norton Unerase or an equivalent program on an SSD? I’d like to hear a first hand account to help confirm (or deny) what the author claims.
- ajphdiv 7y ago
  For the end user it is difficult. Files can be recovered as long garbage collection hasnt happened yet. SSDs require the space to be empty before new data can be written to it. So often times the deleted space is empty. If files are stored in the recycle bin or trash those can still be easily recovered. It was once explained to me that writing to a HDD is like painting. Where new data can just be painted over the top of the old discarded data. While writing to an SSD is like writing on a chalk board where the old data has to be removed before you can write over it.
- toast0 7y ago
  It would likely depend of if the filesystem uses TRIM to notify the SSD of deleted sectors. If it doesn't, the deleted files should retain their data until the filesystem reuses the sector, as normal. If trim is used, the SSD doesn't have to retain the data, but it doesn't necessarily make it unreadable immediately, there are many implementation strategies for trim.
  - wtallis 7y ago
    Enterprise SSDs for the most part promise that reading a trimmed range of the drive will return zeros. Consumer SSDs usually won't make that strong of a guarantee, so that if you're running enterprise software that requires this behavior you have to pay extra for enterprise drives. As originally formulated, the TRIM command was supposed to be more of a hint that the SSD could ignore if it was too busy or if the TRIMMed block was too small for the drive to do anything useful with.
- 7y ago
plughs 7y ago
I found these helpful. The first one has a link to a video which is pretty ELI5 https://flashdba.com/2015/01/09/understanding-flash-floating-gates-and-wear/ https://flashdba.com/2015/01/09/understanding-flash-floating... https://www.youtube.com/watch?v=s7JLXs5es7I https://www.youtube.com/watch?v=s7JLXs5es7I
mjw1007 7y ago
I hoped it would discuss how SSDs cope with sudden power loss, but it doesn't seem to. I remember this page but I don't know of a modern update: http://lkcl.net/reports/ssd_analysis.html http://lkcl.net/reports/ssd_analysis.html These days, if I want an SSD for my desktop and want to minimise the chance I have a disk problem and have to restore from backup, would I be better off with one "data centre" drive (eg Intel D3-S4510), or two mirrored "consumer" drives (perhaps from two different manufacturers)? The prices look similar either way.
- bb88 7y ago
  The big problem with one data center drive is that if that goes bad, you still lose all your data. You're assuming their marketing MTBF is correct. They do make NVME raid solutions now -- with the advantage being that NVME can be faster than SATA. And there are various price points for the NVME drives depending upon speed. This one from 2018 (not sure if it has full raid or uses VROC)(EDIT: it requires software raid) https://www.pcworld.com/article/3297970/highpoint-7101a-pcie-nvme-raid-card-review.html https://www.pcworld.com/article/3297970/highpoint-7101a-pcie... This one is cheaper but relies upon Intel VROC (which has been hard to get working on some mobo's apparently) https://www.amazon.com/ASUS-M-2-X16-V2-Threadripper/dp/B07NQBQB6Z/ https://www.amazon.com/ASUS-M-2-X16-V2-Threadripper/dp/B07NQ... In either case you're looking at max throughput of 11 gigabytes per second, which is roughly 20 times faster than SATA 3's 6 gigabits per second.
  - mxfh 7y ago
    As stated, support for that one is not guaranteed, you need to first figure, out if your motherboard configuration supports lane bifucartion, otherwise it won't work, or only one of the installed keys. Cards with PLX switches are are way to fix this if you cant upgrade your whole hardware, but the price point is a multiple of simple bifurcation cards, since you have to integrate a whole PCI switch on-card. The architecture diagrams here are quite helpful: https://www.qnap.com/en-us/product/qm2-m.2ssd https://www.qnap.com/en-us/product/qm2-m.2ssd
  - wtallis 7y ago
    Almost all NVMe RAID products—including both that you've linked to—are software RAID schemes. So if you're on Linux and already have access to competent software RAID, you should only concern yourself with what's necessary to get the drives connected to your system. In the case of two drives, most recent desktop motherboards already have the slots you need, and multi-drive riser cards are unnecessary.
tjoff 7y ago
> The OCZ Myth: [...] with one overwrite pass of zeroes [...] A sort of RETRIM before that was invented. It wasn't a myth, that was the idea all along. > SSD Defragmentation [...] An important factor left out is wear leveling, it doesn't make as much sense to arrange data in "file-system-order" when the bits on the drive move around.
mxfh 7y ago
I get that it's meant to be subjective, but the premise makes it clear that it's an ongoing knowlegde collection, but for that some sources would be nice. Currently my troubles with SSDs in PC/Server/NAS environment are a somewhat more practical, more about compatibility NVMe/SATA, M.2 key types, PCIe port bifurcation support vs PLX switches, none of them are even mentioned. Advice for this is notoriously hard to find, resorting to trusting rare and random forum posts is my state of knowledge progress there.
- shifto 7y ago
  On forums like ServeTheHome all this information is readily available. It's spread between some guides and posts but if you have a specific question those people will be able to answer it most of the time.
Rafuino 7y ago
The author forgot about non-NAND SSDs (e.g. Optane SSDs). There's no garbage collection to worry about, for example.
- wtallis 7y ago
  Optane SSDs do need wear leveling. What makes it much simpler is that there isn't the mismatch between small-ish NAND pages and massive NAND erase blocks, so you don't have to suffer from the really horribly large read-modify-write cycles.
0xff00ffee 7y ago
Excellent summary. One thing he left off: some SSDs continue to copy/erase blocks even if there is nothing new to write because multi-level cell state does degrade over time. There is a concern that some MLC drives will suffer bit corruption over time if not regularly power up to allow this in the background. Citation needed: I only recall this when I was interviewing to work for Western Digital many years ago.
- wtallis 7y ago
  This problem was most prominent right before the switch to 3D NAND, when planar NAND dimensions were at their smallest and the consumer market had already mostly switched over to 3 bit per cell TLC rather than 2bpc MLC. In the worst case, we were down to about 8 electrons difference between cell voltage states. That's now been relaxed by 3D NAND allowing for larger cell sizes, and most 3D NAND also switched from floating-gate to charge-trap cell design so leakage is less of an issue. Nowadays, data retention in SSDs is only a concern toward the end of their lifespan (as measured by write endurance), and it's probably inadvisable for the SSD to start doing background data scrubbing until the raw read error rate starts climbing.
  - 0xff00ffee 7y ago
    Thank you for bringing me up to speed!
lostgame 7y ago
It lost me at Comic Sans, and made me almost ill with any further font and design choices. So I finally kicked it into Reader View, only to find a lot of questionable spelling and grammar issues. These kinds of basic things go a long way to making an article valuable.
sn 7y ago
Things I've learned from using SSDs at prgmr: Since the firmware is more complicated than hard drives, they are way more likely to brick themselves completely instead of a graceful degradation. Manufacturers can also have nasty firmware bugs like https://www.techpowerup.com/261560/hp-enterprise-ssd-firmware-bug-causes-them-to-fail-at-32-768-hours-of-use-fix-released https://www.techpowerup.com/261560/hp-enterprise-ssd-firmwar... . I'd recommend using a mix of SSDs at different lifetimes, and/or different manufacturers, in a RAID configuration. How different manufacturers deal with running SMART tests under load drastically varies. Samsung tests always take the same amount of time. The length of Intel tests vary depending on load. Micron SMART tests get stuck if they are under constant load. Seagate SMART tests appear to report being at 90% done or complete, but the tests do actually run. Different SSDs also are more or less tolerant to power changes. Micron SSDs are prone to resetting when a hard disk is inserted in the same backplane power domain, and we have to isolate them accordingly. Manual overprovisioning is helpful when you aren't able to use TRIM. What a drive does with secure-erase-enhanced can be different too. Some drives only change the encryption key and then return garbage on read. Some additionally wipe the block mappings so that reads return 0.
- wtallis 7y ago
  Have you found any real value to instructing in-service SSDs to run SMART self-tests, vs simply observing and tracking the SMART indicators over time?
  - sn 7y ago
    It's not as valuable in and of itself as monitoring SMART counters. We've only had a single SSD report failures during a long test, and it also reported an uncorrectable error. However, not finishing the test is a good proxy for if a drive is overloaded and less able to perform routine housekeeping.
- ryanmercer 7y ago
  >I'd recommend using a mix of SSDs at different lifetimes, and/or different manufacturers, in a RAID configuration. Oof, that's extremely obvious yet it never crossed my mind. Nice tip!
vira28 7y ago
thanks for writing. Is it me or someone else noticed that the font is too tough to read?
- Thessdauthor 7y ago
  I am the author of kcall.co.uk/ssd/index.html and I trust that the site is now back up and running without further 'sleeping' episodes. The reason for the interrupts was, as some have surmised, that the site was mentioned here and in other places and that caused a surge in hits that exceeded the host's usage limit. Whilst I am pleased that so many people have shown an interest in my efforts it does mean that I have had to move the site to a more generous host. It is apparent from the first sentence and the website itself that this article is, or arises from, tha musings of shall we say an amateur. It's not professional because I'm not a professional. Whilst some of the comments in this thread are helpful, some are baffling. To the person who couldn't get past the header bacause Comic Sans offended him, the following 8058 words were in Century Gothic, which is perhaps not so offensive. I have however accepted the comments on readability and changed the header and all the rest of the text to Calibri, made the font larger, and changed the line spacing to web standards. I hope it is now more soothing and readable. As for the claim that the text is 'spoiled by having dozens of typos and spelling mistakes' I'm not sure what he is reading (or smoking). Apart from the removal of one stray colon, the current text is entirely unchanged - it is the same file. I don't claim to be perfect and I'm sure that in such a large essay there are some typos and other errors, but to say that there are dozens is patently untrue. Just let me know where they are and I will happily correct them. Apart from all that, I am surprised that there are so few comments about the veracity or otherwise of the contents. I had great difficulty in finding material that was up to date and actuially delved into the technicalities of SSDs, without baffling me - as much did. I shall end by thanking those who found the article interesting and I hope that some of it at least has been of some help.
Stratoscope 7y ago
"Website is sleeping" 000webhost lives up to its name! In the meantime: https://archive.is/20200115095916/http://kcall.co.uk/ssd/index.html https://archive.is/20200115095916/http://kcall.co.uk/ssd/ind...
- geek_at 7y ago
  strange. archive.is is resolving to 127.0.0.5 for me even if I do nslookup archive.is 1.1.1.1 Server: one.one.one.one Address: 1.1.1.1 Name: archive.is Address: 127.0.0.5
  - AdamJacobMuller 7y ago
    It's some feud between the 1.1.1.1 operators and the archive.is operator. I never really cared enough to figure out who was at fault. Probably both of them. edit; here you go: https://jarv.is/notes/cloudflare-dns-archive-is-blocked/ https://jarv.is/notes/cloudflare-dns-archive-is-blocked/
  - Stratoscope 7y ago
    Here are a couple more mirrors: https://webcache.googleusercontent.com/search?q=cache:http%3A%2F%2Fkcall.co.uk%2Fssd%2Findex.html https://webcache.googleusercontent.com/search?q=cache:http%3... http://cc.bingj.com/cache.aspx?q=url:http%3A%2F%2Fkcall.co.uk%2Fssd%2Findex.html&d=4855675377620122&mkt=en-WW&setlang=en-US&w=1oy47Uw_sjWgG1ZfbJ2ltPcVHIfT1mID http://cc.bingj.com/cache.aspx?q=url:http%3A%2F%2Fkcall.co.u... Hopefully one of them will work for you.
    - lucb1e 7y ago
      Those will stop working some time soon. Long-term mirror: https://web.archive.org/web/20200115163630/http://kcall.co.uk/ssd/index.html https://web.archive.org/web/20200115163630/http://kcall.co.u...
  - lxe 7y ago
    Do you have AT&T? Check this out: https://www.google.com/search?q=At%26t+1.1.1.1+blocking https://www.google.com/search?q=At%26t+1.1.1.1+blocking
StavrosK 7y ago
Mirror: http://archive.is/K9SFI http://archive.is/K9SFI
- exikyut 7y ago
  archive.is links have not been working for me for a while now. I wasn't sure if something was wrong with the site or what, but seeing this link posted here convinced me to do a bit of digging. I have news: 1.1.1.1, aka CloudFlare's DNS, resolves archive.is to 127.0.0.5. Live proof: https://digwebinterface.com/?hostnames=archive.is&useresolver=1.1.1.1 https://digwebinterface.com/?hostnames=archive.is&useresolve... Archived proof: http://archive.is/utJfW http://archive.is/utJfW Of course the moment I changed 1.1.1.1 to 8.8.8.8 in resolv.conf I was able to access the site again.
  - leesalminen 7y ago
    1.1.1.1 has been acting up for me recently as well but haven’t had a chance to investigate. Glad to hear it’s not just me
    - deleted 7y ago
      [deleted]
    - vsl 7y ago
      It’s really archive.is acting up here...
  - uj8efdkjfdshf 7y ago
    It's over a disagreement between Cloudflare and the owner of Archive.is regarding the forwarding of EDNS metadata. See here: https://news.ycombinator.com/item?id=19828317 https://news.ycombinator.com/item?id=19828317
  - StavrosK 7y ago
    Yes, there was some discussion a while ago here, CloudFlare says archive.is won't reply with the correct DNS to them and the author of archive.is says CloudFlare isn't passing the correct information to his server, and he refuses to change it. I don't remember details, I'm afraid, but it is an unfortunate situation.
lousken 7y ago
site doesn't work - archive link https://archive.is/K9SFI https://archive.is/K9SFI
pettycashstash2 7y ago
Says website is sleeping? did he go over the bandwidth limit?
deleted 7y ago
[deleted]

352 points by classified 7y ago

The part about filesystems is slightly incorrect. > The way the file system handles this is incompatible with the workings of NAND flash. That's true of most conventional filesystems, but log-structured filesystems are much more flash-friendly. That's why there has been a resurgence of interest in them, and also why a typical flash translation layer bears a striking resemblance to a log-structured FS. There are also flash-specific filesystems. > to an HDD all sectors are the same. This is not true because of bad blocks. Every disk has a reserve of blocks that can be remapped in place of a detected bad block, transparently, much like flash blocks are remapped. Beyond that, it's also useful for a disk to know which blocks are not in use so it can return all zeroes without actually hitting the media. There are special flags to force media access and commands to physically zero a block, for the cases where those are needed, but often they're not. Trim/discard actually gets pretty complicated, especially when things like RAID and virtual block devices are involved.

rckoepke 7y ago
> to an HDD all sectors are the same. Also I believe some humans, and (filesystems?) intentionally stored certain data towards the inside/outside of the HDD because the simple cylinder geometry allowed faster reads in those regions. However, I'm not seeing conclusive proof that modern HDDs show performance variation with respect to radius.
- monocasa 7y ago
  Io schedulers would also understand the geometry and reorder commands to minimize seek distance (and therefore seek time).
  - notacoward 7y ago
    I used to do some of that a long time ago, and it was actually kind of fun. There's a whole different set of optimizations to do on flash, both for performance and (more importantly in many use cases) power efficiency, and I think that would be fun too.
  - tzs 7y ago
    At some point, I think around the mid '90s or early 2000s but maybe earlier, though, seek time was fast enough on widely available drives that on average for random access you spent about as much time on the right cylinder waiting for the sector you wanted to rotate to under the head as you did on seeking to the right cylinder. You could get some decent gains then if you made your scheduler take rotation into account. A long seek that arrived just before the target sector came under the head could be faster than a short seek that would arrive just after the sector passed the head. On the other hand, taking rotation into account could make the scheduler quite a bit more complex. You needed a model that could predict seek time well, and you needed to know the angular position of each sector in its cylinder. I don't think that there were any drives that would tell you this. SCSI drives wouldn't even tell you the geometry. IDE drives would tell you a geometry, but it didn't necessarily have anything to do with the actual geometry of the drive. At the time I worked at a company that was working on disk performance enhancement software (e.g., drivers with better scheduling, utilities that would log disk accesses and then rearrange data on the disk so that the I/O patterns in the logs would be faster [1], and that sort of thing). We had a program that could get the real disk geometry. It did so by doing a lot of random I/O and looking at the timing of when the results came back. If there were no disk cache, this would be fairly easy. (Well, it didn't necessarily get the real geometry, but rather a purported geometry and seek and rotational characteristics that could predict I/O time well). For instance, read some random sector T, then read another random sector, then read T again. Look at the time difference between when you started getting data back on the two reads of T. This should be a multiple of the rotation time. If the disk has caching that can still work but you need to read a lot of random sectors between the two reads of T to try to get the first read out of the cache. Anyway, we had to give up on that approach because the program to analyze the disk took a few days of constant I/O to finish. Management decided that most consumers would not put up with such a long setup procedure. [1] Yes, that could mean that it would purposefully make files more fragmented. A fairly common pattern was for a program to open a bunch of data files and read a header from each. E.g., some big GUI programs would do that for a large number of font files. Arranging that program and those data files on disk so that you have the program code that gets loaded before the header reads, then the headers of all the files, and then the rest of the file data, could give you a nice speed boost. The flaw in this method is that, to use the above example, if another big GUI program also uses those same font files, the layout that makes the first program go fast might suck for the second program. If you've got a computer that you mostly only use for one task, though, it can be a viable approach.
- wtallis 7y ago
  > However, I'm not seeing conclusive proof that modern HDDs show performance variation with respect to radius. They definitely still do; it's fundamental to drives that run at fixed RPM but maintain high areal density across the entire platter. One of the 1TB drives I have lying around does about 183MB/s at the beginning of the disk, 148MB/s in the middle, and 97MB/s at the end.
  - ascar 7y ago
    I'm really interested in how you benchmarked that? Is there a "simple" way to specify the physical location on a HD data should be written to?
    - wtallis 7y ago
      Hard drives generally use a fairly simple linear mapping between LBAs and physical location. Low LBAs are on the outer edge of the platter where transfer speeds are highest, and high LBAs are on the inner edge where speeds are lowest. Unlike SSDs, hard drives don't need wear leveling, so there's no reason to break from that pattern except in the relatively rare instance of damaged sectors. At smaller scales, the layout of individual tracks can vary quite a bit, but that doesn't have as much impact on overall sequential transfer speed. See http://blog.stuffedcow.net/2019/09/hard-disk-geometry-microbenchmarking/ http://blog.stuffedcow.net/2019/09/hard-disk-geometry-microb...
      - zozbot234 7y ago
        Your link was discussed @ https://news.ycombinator.com/item?id=21492345 https://news.ycombinator.com/item?id=21492345
        wtallis 7y ago
        Yep. I found it interesting enough to bookmark, and now I tend to pull it up when somebody wants to know how hard drives really work.
      - mehrdadn 7y ago
- kabdib 7y ago
  This is still very much true, to the point that game consoles will reserve areas of rotating media for different purposes, depending on speed (e.g., disk caches on the Xbox are on the outer tracks). For optical media the layout of assets can be critical to the user experience. You probably want your startup assets located on the outer, faster tracks.
- marcosdumay 7y ago
  People reduce the number of cylinders to keep seek time under control. It just happens that if you are going to choose what cylinders to use, you better choose the inner ones that are slightly faster.
  - Hackbraten 7y ago
    Wouldn’t the outer ones be slightly faster in terms of both transfer rate and head travel time?
- CrazyStat 7y ago
  I remember reading a blog post ca. 2005, before SSDs were common, in which the author explained how he had identified all the files his system needed to boot and the order in which they were read from disk, and then contrived to put them all in the correct order in a contiguous block at the start of the drive to eliminate seeks during boot entirely. This apparently shaved a couple seconds off his boot time.
  - dspillett 7y ago
    There are automated solutions to this. Windows has had similar tricks built in (at various levels of cleverness, depending on the Windows variant you run) for a while now: https://en.wikipedia.org/wiki/Prefetcher https://en.wikipedia.org/wiki/Prefetcher No doubt there are similar options for Linux and other OSs. A similar trick I've seen is using a small SSD as a manual cache: copy in the files that need to be fast and mount it with the larger filesystem (on slower drives) as a unioned filesystem. Though just using block device based automatic cacheing may be easier and safer than rolling your own mess. There are a few options for Linux though some are not currently maintained (see https://serverfault.com/questions/969302/linux-ssd-as-hdd-cache https://serverfault.com/questions/969302/linux-ssd-as-hdd-ca... amongst other places), and some IO controllers support it directly (even some motherboards have this built in) without needing to bother your OS with the details at all.
guenthert 7y ago
> but log-structured filesystems are much more flash-friendly. That's why there has been a resurgence of interest in them I could swear I read such in 2010 -- did that actually happen in the last ten years?
- osmarks 7y ago
  f2fs (https://en.wikipedia.org/wiki/F2FS https://en.wikipedia.org/wiki/F2FS) is now popular for Android-based mobile devices.
  - guenthert 7y ago
    That counts, thanks!
rodgerd 7y ago
He's also inaccurate in characterizing all cells as "1 or 0"; digital electronics have always been a cut-off of an underlying analogue value (such as voltage).
- outworlder 7y ago
  They explain this in more detail later on. See "SSD Reads" and "interpreting the results".
namibj 7y ago
I briefly searched for a source of (mostly) FTL-less microSD cards (I'd be fine with them doing wear leveling-related remapping of whole blocks, as long as it's power-loss-safe, but I don't want them to expend any effort on emulating 512/4k random write capability). I didn't find anything but bunnie's blog entry[0] on hacking/re-flashing the firmware on the card's controller. Due to how important it is to have a good FTL, and the pretty much native suitability of log-structured data formats, like those based on RocksDB or InfluxDB (both not unlikely for data handled and stored on RPi-like SBCs), it'd be much better for reliability and performance reasons to let these LSM engines deal with NAND flash's block erase/sector write/sector read behavior. [0]: https://www.bunniestudios.com/blog/?p=3554 https://www.bunniestudios.com/blog/?p=3554

Ariez 7y ago

I was under the impression that if you do not encrypt an SSD from the first use, then any attempt at overwriting with 0s is futile, as well as any other method to securely delete the files. The files will be easily recovered. This guy seems to say the opposite, in that the files are "simply not there anymore", contrary to everything I've read: who's right here?

blattimwind 7y ago
In practice all SSDs are always encrypted because they use the encryption to whiten the data written to them. That's why "Secure Erase" takes less than a second on SSDs, it doesn't erase anything but the key.
- e12e 7y ago
  But AFAIK this isn't true for all solid state storage, like cheap USB memory keys. These would generally also benefit from log structured filesystems - but really should be encrypted too. (which is one great promise of zfs on linux/openzfs - cross-platform encrypted removable storage).
- nemo1618 7y ago
  Interesting! Do you happen to know which encryption algorithm is used? I would think that, if the only goal is whitening (as opposed to robust security), a fairly weak algorithm would be used, or perhaps a strong algorithm with a reduced number of rounds.
  - theandrewbailey 7y ago
    The algorithm probably depends on the drive, but AES-256 is common due to hardware acceleration. Read a SSD spec sheet sometime, it will likely mention it, along with supporting TCG Opal (the self encrypting drive standard).
  - sigstoat 7y ago
    the hardware is going to use AES because their ASIC vendor will have well tested AES IP that they can just throw down on the chip. any other algorithm would require massive development effort for zero benefit. and by using AES they can probably claim to satisfy some security standards that will make their marketing people happier.

rahuldottech 7y ago

There's some interesting content about SSDs and HDDs here: SSDs: https://superuser.com/questions/tagged/ssd?tab=Votes https://superuser.com/questions/tagged/ssd?tab=Votes HDDs: https://superuser.com/questions/tagged/hard-drive?tab=Votes https://superuser.com/questions/tagged/hard-drive?tab=Votes

cattlemansgold 7y ago

There are a few technologies that I’ve tried very earnestly to understand, only to find out that it’s basically black magic and there’s no use in trying to understand it. Those things are modern car transmissions, nuclear reactors, and SSDs.

24gttghh 7y ago
A very basic nuclear reactor can be explained pretty simply I think. You enrich a bunch of let's say uranium. Pack it together in a rod, and put a bunch of those rods in a pond. Those rods have controlled (ideally) nuclear decay from their being in close proximity to other rods which generates a lot of heat, which is transferred to a separate cooling loop that boils water to make steam which drives an electric turbine. Now I'm no nuclear scientist so please be forgiving with that description, but that's how I understand them to work :) I can't even begin to explain how an SSD works, but I know there are no moving parts besides electrons. edit: moved the "(ideally)"
- zzzcpan 7y ago
  You can explain SSD as a controller you send commands to that writes data to and reads from a log structured storage on top of raw flash.
- cattlemansgold 7y ago
  I guess what I meant is that the overall concepts are understandable. Nuclear fuel gets hot, boils water, drives a turbine. For transmissions, different sized gears allow things to turn at different rates. But as soon as I dive into the details, I get lost. How exactly can you control the nuclear decay? How exactly does do the gears in the transmission move around and combine with eachother to create a specific gear ratio? These concepts probably are probably pretty simple for a lot of people, but they just make my head spin.
  - toast0 7y ago
    > How exactly can you control the nuclear decay? That's what the control rods are for. The uranium in one fuel rod in isolation decays at whatever natural rate, which would warm water but not boil it, and placing the rods near each other allows for the decay products (high energy particles) to interact with other fuel rods and induce more rapid decay. The control rods slot in between the fuel rods, and absorb the decay products without inducing further nuclear decay. Usually these are graphite rods. > How exactly does do the gears in the transmission move around and combine with eachother to create a specific gear ratio? It really depends on the specific transmission, a manual transmission is using the shift lever movement to move the gears into place. An automatic transmission most likely uses solenoids to move things (a solenoid is basically a coil of wires around a tube with a moveable metal rod inside, when you put current through the wire, the metal rod is pulled into the tube, you attach the larger thing you want to move to the end of the rod (sometimes with a pivot or what not), and use a spring, another solenoid, or gravity, etc to make the reverse movement. A solenoid by itself gives you linear movement, if you need rotational movement, one way to do that is have a pivot on the end of the solenoid rod, then a rod from there to one end of a clamp on a shaft, then when the solenoid pulls in its rod, the shaft will rotate (this is the basic mechanism for pinball flippers).

fareesh 7y ago

I wonder if anyone here has experienced something similar: I have a Samsung Evo 860 SSD. Sometimes after powering on my desktop the BIOS either "forgets" the drive (sets some other drive as primary boot drive) or doesn't recognize it at all. The non recognition issue goes away after I power off and power on. It's been this way for about 8 months. Happens every 1/15 times I power on. I've heard it may have something to do with "sleep mode" or something like that. I always shutdown via software though.

cptskippy 7y ago
That sounds like the drive is failing to initialize in time. Have you tried enabling a POST test or Boot Delay? I suspect the problem might magically go away if you do.
- fareesh 7y ago
  I'll try it out
leetcrew 7y ago
might be an innocent config issue, but this usually means the drive detection timed out during boot, which can be an early sign of drive failure. one of my old ssds behaved this way for a while before never being detected again. make sure you have anything important backed up.
toast0 7y ago
I had an ssd/system combination that would not communicate at sata 3, and either the bios wouldn't try at sata 2 or it would timeout detection too soon after. If your bios lets you set a sata revision, it's worth a try (mine didn't, but I used a sketchy download of the AMI Bios Editor to adjust the default; I couldn't figure out how to use the editor enough to make the setting visible). And also maybe reseat the cables.
cmurf 7y ago
Is the drive's firmware current? And what about the logic board firmware?

cptskippy 7y ago

The CSS of this site manages to making reading this article equality inconvenient and painful on both Mobile and Desktop.

_hao 7y ago
Same experience for me. Text seems readable and can be easily followed at 250% zoom (at least for me)
amarshall 7y ago
Indeed. Thankfully, Reader View in both Firefox and mobile Safari remedies it.

creeble 7y ago

Anyone know how TRIM works with Linux? I find myself copying entire partitions between SSDs from time to time, is there a utility to clear the destination SSD before copy? Is it possible to do the same for an SD card, so that writing a new Raspi OS to it doesn't do unnecessary garbage collection?

leetcrew 7y ago
I believe all major OSes support trim in a reasonable way now. afaik it needs hardware support though, so idk how it would work for an SD card.
- cm2187 7y ago
  Unless you have a RAID card in the middle.
  - leetcrew 7y ago
    I don't mean to say it wouldn't work, just that I don't know enough about sd cards to say whether it would/should.
    - cm2187 7y ago
      Sorry, I meant for TRIM being passed down to an SSD.
      - leetcrew 7y ago
        oh gotcha. not sure why I thought it was reasonable to think you were talking about a bunch of SD cards in a raid array...
- monocasa 7y ago
  MMC defines trim as an optional command. But maybe to your point I've generally only seen it implemented on eMMC chips, not SD cards.
7y ago

cbdumas 7y ago

I love a minimal, text-only website as much as the next crotchety HN reader, but a little bit of CSS goes a very long way in terms of readability. Edit: Upon further inspection I see that this page was designed to be hard to read. Very curious.

occamrazor 7y ago
Reader view helps a lot, and with such a simple website it is guaranteed to give a good result.
- dr_zoidberg 7y ago
  Not just a lot, it actually makes it readable. However I do feel this article needs a few graphs and diagrams. I've written [0] about data recovery and SSD/Flash storage is a massive beast to tackle even with diagrams to give you a vague idea of what's going on. [0] in old fashioned tree-based paper, unfortunately
  - ShroudedNight 7y ago
    Is there an available method to acquire some of these marked-up dead trees?
- 867-5309 7y ago
  not if you're reading via Materialistic via Android browser with no responsive design or zoom ~,~
  - Flott 7y ago
    In Materialistic you can hit the red round button that appears bottom right of the screen. That way you can zoom. It won't help you much with this article, but it can be handy sometimes.
computator 7y ago
If the author is reading this, I’d like to ask him to please run his article through a spell checker. All the effort to write a nice article is spoiled by having dozens of typos and spelling mistakes.

ololobus 7y ago

"As for writes, at 1 gb a day - far more than my current rate of data use - it would take the same 114 years to reach 40 tb." Maybe it works for author's very specific system or use-case, but on my personal MBP laptop with very occasional usage pattern--some days I do not use it at all during the week--I end up with 10 GB per day of writes on average. That way it will be 11.4 years already, not so many. And I do not do something very disk-expensive on it like torrents downloading or database testing, only some general development task, watching online videos, web surfing, docs, etc.

gowld 7y ago
How much RAM do you have? And how many Chrome tabs?
- ololobus 7y ago
  16 GB of RAM and I use Safari mostly and Firefox from time to time with less than 20 tabs usually (less than 10 in average, I guess). I check Data written in the Activity monitor — Disk.
ls612 7y ago
I have an NVME SSD as a boot drive in my desktop and three years and change in (say 1200 days) it’s used 40tb of write (out of an advertised 400tb endurance so I’m not too worried). That works out to about 30gb of writes per day, which seems about right for medium to heavy use. I guess what I’m saying is that for modern SSDs I don’t think write endurance is a binding constraint in most cases.
- wtallis 7y ago
  For client/consumer SSDs, most vendors seem to view 50GB/day as plenty of write endurance for mainstream, non-enthusiast products. Virtually all retail consumer SSDs have a warranty that covers at least that much, and usually several times more for larger drive capacities (since write endurance is more often specified in drive writes per day).

computator 7y ago

> Deleted file recovery on a modern SSD is next to impossible for the end user OK, he’s talking about SSDs, but I want to mention that I’ve easily recovered many large deleted files from SD or micro-SD cards (formatted as FAT32 or exFAT) using Norton Unerase or an equivalent utility. Are the controllers for SSDs that different from the controllers for SD cards? Has anyone tried Norton Unerase or an equivalent program on an SSD? I’d like to hear a first hand account to help confirm (or deny) what the author claims.

ajphdiv 7y ago
For the end user it is difficult. Files can be recovered as long garbage collection hasnt happened yet. SSDs require the space to be empty before new data can be written to it. So often times the deleted space is empty. If files are stored in the recycle bin or trash those can still be easily recovered. It was once explained to me that writing to a HDD is like painting. Where new data can just be painted over the top of the old discarded data. While writing to an SSD is like writing on a chalk board where the old data has to be removed before you can write over it.
toast0 7y ago
It would likely depend of if the filesystem uses TRIM to notify the SSD of deleted sectors. If it doesn't, the deleted files should retain their data until the filesystem reuses the sector, as normal. If trim is used, the SSD doesn't have to retain the data, but it doesn't necessarily make it unreadable immediately, there are many implementation strategies for trim.
- wtallis 7y ago
  Enterprise SSDs for the most part promise that reading a trimmed range of the drive will return zeros. Consumer SSDs usually won't make that strong of a guarantee, so that if you're running enterprise software that requires this behavior you have to pay extra for enterprise drives. As originally formulated, the TRIM command was supposed to be more of a hint that the SSD could ignore if it was too busy or if the TRIMMed block was too small for the drive to do anything useful with.
7y ago

plughs 7y ago

I found these helpful. The first one has a link to a video which is pretty ELI5 https://flashdba.com/2015/01/09/understanding-flash-floating-gates-and-wear/ https://flashdba.com/2015/01/09/understanding-flash-floating... https://www.youtube.com/watch?v=s7JLXs5es7I https://www.youtube.com/watch?v=s7JLXs5es7I

mjw1007 7y ago

I hoped it would discuss how SSDs cope with sudden power loss, but it doesn't seem to. I remember this page but I don't know of a modern update: http://lkcl.net/reports/ssd_analysis.html http://lkcl.net/reports/ssd_analysis.html These days, if I want an SSD for my desktop and want to minimise the chance I have a disk problem and have to restore from backup, would I be better off with one "data centre" drive (eg Intel D3-S4510), or two mirrored "consumer" drives (perhaps from two different manufacturers)? The prices look similar either way.

bb88 7y ago
The big problem with one data center drive is that if that goes bad, you still lose all your data. You're assuming their marketing MTBF is correct. They do make NVME raid solutions now -- with the advantage being that NVME can be faster than SATA. And there are various price points for the NVME drives depending upon speed. This one from 2018 (not sure if it has full raid or uses VROC)(EDIT: it requires software raid) https://www.pcworld.com/article/3297970/highpoint-7101a-pcie-nvme-raid-card-review.html https://www.pcworld.com/article/3297970/highpoint-7101a-pcie... This one is cheaper but relies upon Intel VROC (which has been hard to get working on some mobo's apparently) https://www.amazon.com/ASUS-M-2-X16-V2-Threadripper/dp/B07NQBQB6Z/ https://www.amazon.com/ASUS-M-2-X16-V2-Threadripper/dp/B07NQ... In either case you're looking at max throughput of 11 gigabytes per second, which is roughly 20 times faster than SATA 3's 6 gigabits per second.
- mxfh 7y ago
  As stated, support for that one is not guaranteed, you need to first figure, out if your motherboard configuration supports lane bifucartion, otherwise it won't work, or only one of the installed keys. Cards with PLX switches are are way to fix this if you cant upgrade your whole hardware, but the price point is a multiple of simple bifurcation cards, since you have to integrate a whole PCI switch on-card. The architecture diagrams here are quite helpful: https://www.qnap.com/en-us/product/qm2-m.2ssd https://www.qnap.com/en-us/product/qm2-m.2ssd
- wtallis 7y ago
  Almost all NVMe RAID products—including both that you've linked to—are software RAID schemes. So if you're on Linux and already have access to competent software RAID, you should only concern yourself with what's necessary to get the drives connected to your system. In the case of two drives, most recent desktop motherboards already have the slots you need, and multi-drive riser cards are unnecessary.

tjoff 7y ago

> The OCZ Myth: [...] with one overwrite pass of zeroes [...] A sort of RETRIM before that was invented. It wasn't a myth, that was the idea all along. > SSD Defragmentation [...] An important factor left out is wear leveling, it doesn't make as much sense to arrange data in "file-system-order" when the bits on the drive move around.

mxfh 7y ago

I get that it's meant to be subjective, but the premise makes it clear that it's an ongoing knowlegde collection, but for that some sources would be nice. Currently my troubles with SSDs in PC/Server/NAS environment are a somewhat more practical, more about compatibility NVMe/SATA, M.2 key types, PCIe port bifurcation support vs PLX switches, none of them are even mentioned. Advice for this is notoriously hard to find, resorting to trusting rare and random forum posts is my state of knowledge progress there.

shifto 7y ago
On forums like ServeTheHome all this information is readily available. It's spread between some guides and posts but if you have a specific question those people will be able to answer it most of the time.

Rafuino 7y ago

The author forgot about non-NAND SSDs (e.g. Optane SSDs). There's no garbage collection to worry about, for example.

wtallis 7y ago
Optane SSDs do need wear leveling. What makes it much simpler is that there isn't the mismatch between small-ish NAND pages and massive NAND erase blocks, so you don't have to suffer from the really horribly large read-modify-write cycles.

0xff00ffee 7y ago

Excellent summary. One thing he left off: some SSDs continue to copy/erase blocks even if there is nothing new to write because multi-level cell state does degrade over time. There is a concern that some MLC drives will suffer bit corruption over time if not regularly power up to allow this in the background. Citation needed: I only recall this when I was interviewing to work for Western Digital many years ago.

wtallis 7y ago
This problem was most prominent right before the switch to 3D NAND, when planar NAND dimensions were at their smallest and the consumer market had already mostly switched over to 3 bit per cell TLC rather than 2bpc MLC. In the worst case, we were down to about 8 electrons difference between cell voltage states. That's now been relaxed by 3D NAND allowing for larger cell sizes, and most 3D NAND also switched from floating-gate to charge-trap cell design so leakage is less of an issue. Nowadays, data retention in SSDs is only a concern toward the end of their lifespan (as measured by write endurance), and it's probably inadvisable for the SSD to start doing background data scrubbing until the raw read error rate starts climbing.
- 0xff00ffee 7y ago
  Thank you for bringing me up to speed!

lostgame 7y ago

It lost me at Comic Sans, and made me almost ill with any further font and design choices. So I finally kicked it into Reader View, only to find a lot of questionable spelling and grammar issues. These kinds of basic things go a long way to making an article valuable.

sn 7y ago

Things I've learned from using SSDs at prgmr: Since the firmware is more complicated than hard drives, they are way more likely to brick themselves completely instead of a graceful degradation. Manufacturers can also have nasty firmware bugs like https://www.techpowerup.com/261560/hp-enterprise-ssd-firmware-bug-causes-them-to-fail-at-32-768-hours-of-use-fix-released https://www.techpowerup.com/261560/hp-enterprise-ssd-firmwar... . I'd recommend using a mix of SSDs at different lifetimes, and/or different manufacturers, in a RAID configuration. How different manufacturers deal with running SMART tests under load drastically varies. Samsung tests always take the same amount of time. The length of Intel tests vary depending on load. Micron SMART tests get stuck if they are under constant load. Seagate SMART tests appear to report being at 90% done or complete, but the tests do actually run. Different SSDs also are more or less tolerant to power changes. Micron SSDs are prone to resetting when a hard disk is inserted in the same backplane power domain, and we have to isolate them accordingly. Manual overprovisioning is helpful when you aren't able to use TRIM. What a drive does with secure-erase-enhanced can be different too. Some drives only change the encryption key and then return garbage on read. Some additionally wipe the block mappings so that reads return 0.

wtallis 7y ago
Have you found any real value to instructing in-service SSDs to run SMART self-tests, vs simply observing and tracking the SMART indicators over time?
- sn 7y ago
  It's not as valuable in and of itself as monitoring SMART counters. We've only had a single SSD report failures during a long test, and it also reported an uncorrectable error. However, not finishing the test is a good proxy for if a drive is overloaded and less able to perform routine housekeeping.
ryanmercer 7y ago
>I'd recommend using a mix of SSDs at different lifetimes, and/or different manufacturers, in a RAID configuration. Oof, that's extremely obvious yet it never crossed my mind. Nice tip!

vira28 7y ago

thanks for writing. Is it me or someone else noticed that the font is too tough to read?

Thessdauthor 7y ago
I am the author of kcall.co.uk/ssd/index.html and I trust that the site is now back up and running without further 'sleeping' episodes. The reason for the interrupts was, as some have surmised, that the site was mentioned here and in other places and that caused a surge in hits that exceeded the host's usage limit. Whilst I am pleased that so many people have shown an interest in my efforts it does mean that I have had to move the site to a more generous host. It is apparent from the first sentence and the website itself that this article is, or arises from, tha musings of shall we say an amateur. It's not professional because I'm not a professional. Whilst some of the comments in this thread are helpful, some are baffling. To the person who couldn't get past the header bacause Comic Sans offended him, the following 8058 words were in Century Gothic, which is perhaps not so offensive. I have however accepted the comments on readability and changed the header and all the rest of the text to Calibri, made the font larger, and changed the line spacing to web standards. I hope it is now more soothing and readable. As for the claim that the text is 'spoiled by having dozens of typos and spelling mistakes' I'm not sure what he is reading (or smoking). Apart from the removal of one stray colon, the current text is entirely unchanged - it is the same file. I don't claim to be perfect and I'm sure that in such a large essay there are some typos and other errors, but to say that there are dozens is patently untrue. Just let me know where they are and I will happily correct them. Apart from all that, I am surprised that there are so few comments about the veracity or otherwise of the contents. I had great difficulty in finding material that was up to date and actuially delved into the technicalities of SSDs, without baffling me - as much did. I shall end by thanking those who found the article interesting and I hope that some of it at least has been of some help.

Stratoscope 7y ago

"Website is sleeping" 000webhost lives up to its name! In the meantime: https://archive.is/20200115095916/http://kcall.co.uk/ssd/index.html https://archive.is/20200115095916/http://kcall.co.uk/ssd/ind...

geek_at 7y ago
strange. archive.is is resolving to 127.0.0.5 for me even if I do nslookup archive.is 1.1.1.1 Server: one.one.one.one Address: 1.1.1.1 Name: archive.is Address: 127.0.0.5
- AdamJacobMuller 7y ago
  It's some feud between the 1.1.1.1 operators and the archive.is operator. I never really cared enough to figure out who was at fault. Probably both of them. edit; here you go: https://jarv.is/notes/cloudflare-dns-archive-is-blocked/ https://jarv.is/notes/cloudflare-dns-archive-is-blocked/
- Stratoscope 7y ago
  Here are a couple more mirrors: https://webcache.googleusercontent.com/search?q=cache:http%3A%2F%2Fkcall.co.uk%2Fssd%2Findex.html https://webcache.googleusercontent.com/search?q=cache:http%3... http://cc.bingj.com/cache.aspx?q=url:http%3A%2F%2Fkcall.co.uk%2Fssd%2Findex.html&d=4855675377620122&mkt=en-WW&setlang=en-US&w=1oy47Uw_sjWgG1ZfbJ2ltPcVHIfT1mID http://cc.bingj.com/cache.aspx?q=url:http%3A%2F%2Fkcall.co.u... Hopefully one of them will work for you.
  - lucb1e 7y ago
    Those will stop working some time soon. Long-term mirror: https://web.archive.org/web/20200115163630/http://kcall.co.uk/ssd/index.html https://web.archive.org/web/20200115163630/http://kcall.co.u...
- lxe 7y ago
  Do you have AT&T? Check this out: https://www.google.com/search?q=At%26t+1.1.1.1+blocking https://www.google.com/search?q=At%26t+1.1.1.1+blocking

StavrosK 7y ago

Mirror: http://archive.is/K9SFI http://archive.is/K9SFI

exikyut 7y ago
archive.is links have not been working for me for a while now. I wasn't sure if something was wrong with the site or what, but seeing this link posted here convinced me to do a bit of digging. I have news: 1.1.1.1, aka CloudFlare's DNS, resolves archive.is to 127.0.0.5. Live proof: https://digwebinterface.com/?hostnames=archive.is&useresolver=1.1.1.1 https://digwebinterface.com/?hostnames=archive.is&useresolve... Archived proof: http://archive.is/utJfW http://archive.is/utJfW Of course the moment I changed 1.1.1.1 to 8.8.8.8 in resolv.conf I was able to access the site again.
- leesalminen 7y ago
  1.1.1.1 has been acting up for me recently as well but haven’t had a chance to investigate. Glad to hear it’s not just me
  - deleted 7y ago
    [deleted]
  - vsl 7y ago
    It’s really archive.is acting up here...
- uj8efdkjfdshf 7y ago
  It's over a disagreement between Cloudflare and the owner of Archive.is regarding the forwarding of EDNS metadata. See here: https://news.ycombinator.com/item?id=19828317 https://news.ycombinator.com/item?id=19828317
- StavrosK 7y ago
  Yes, there was some discussion a while ago here, CloudFlare says archive.is won't reply with the correct DNS to them and the author of archive.is says CloudFlare isn't passing the correct information to his server, and he refuses to change it. I don't remember details, I'm afraid, but it is an unfortunate situation.

lousken 7y ago

site doesn't work - archive link https://archive.is/K9SFI https://archive.is/K9SFI

pettycashstash2 7y ago

Says website is sleeping? did he go over the bandwidth limit?

deleted 7y ago

[deleted]