13 ms·

Everything I know about SSDs (2019)

207 points by fagnerbrack 3y ago

idle_cycles 3y ago
Two wonderful papers that are relevant: 1) https://pages.cs.wisc.edu/~jhe/eurosys17-he.pdf https://pages.cs.wisc.edu/~jhe/eurosys17-he.pdf 2) https://www.usenix.org/system/files/hotstorage19-paper-wu-kan.pdf https://www.usenix.org/system/files/hotstorage19-paper-wu-ka...
dang 3y ago
Discussed at the time: Everything I Know About SSDs - https://news.ycombinator.com/item?id=22054600 https://news.ycombinator.com/item?id=22054600 - Jan 2020 (185 comments)
eatonphil 3y ago
You might also be interested in this AMA we held on r/databasedevelopment with two NVMe developers from Samsung. https://www.reddit.com/r/databasedevelopment/comments/1afpezz/samsung_nvme_developers_ama/ https://www.reddit.com/r/databasedevelopment/comments/1afpez...
- thadt 3y ago
  Thanks for hosting (and posting it here). I was reading the "What Modern NVMe Storage Can Do..." paper just yesterday, and this was a great followup.
jmbwell 3y ago
I always thought 'flash' was a holdover from early reprogrammable ROM technology, non-volatile memory that you could erase by literally flashing a literal flashbulb over a little window on the chip. I would've sworn in a court of law that I recall this being called "flashable" memory, that erasing it was called "flashing" it, and reprogramming it in general was called "reflashing" in a sort of synecdoche. And I'd have assumed that this was the fundamental origin of what became _electronically_ erasable ROM (EEPROM), which led to all the various NVRAM technologies we have now, with "flashing" sticking as the term for reprogramming it, even after you could do it electronically. It looks like the story these days is that someone at Toshiba thought up the name out of the blue. I'm skeptical!
- sgerenser 3y ago
  I never heard of using a literal flash to reprogram EPROMs, but this wikipedia entry[1] makes your story for the origin of the term “Flashing” seem likely EPROMs are easily recognizable by the transparent fused quartz (or on later models resin) window on the top of the package, through which the silicon chip is visible, and which permits exposure to ultraviolet light during erasing. [1] https://en.m.wikipedia.org/wiki/EPROM https://en.m.wikipedia.org/wiki/EPROM
  - jmbwell 3y ago
    It does look like all the references I can find point to engineers at Toshiba in 1980 coining the name, although Google ngrams shows some references to "flash EPROM" prior to 1980, so I can't help but wonder if the idea existed at least in some form prior to 1980
    - monocasa 3y ago
      Looking into it, those references look miscategorized. It's some Zambian national report that's talking about 0.18um processes (aka, 180nm) in the same paragraph, which wouldn't have come out until the very late 90s.
  - radicalbyte 3y ago
    These things were used in a number of computers from the early 90s, I have a few lying around for my Amiga.
  - hex4def6 3y ago
    Heard from a greybeard that they had a demo night in university (this was probably early 80s), and one of the demos was was some sort of path-finding rover robot. Of course, it ended up being one of those projects that had show stopping bugs up until a few hours before showtime. During demo night, it was a big hit, until a stray camera flash got (un)lucky and wiped the microcontroller's EEPROM... Feels a little apocryphal (I'd assume most flashes have / had UV filters?)
password4321 3y ago
I've heard SSDs are more likely to "fail fast". Can anyone recommended utilities that monitor and warn before SSD failures?
- caseyf 3y ago
  For NVMe, if you get the SMART data with smartmontools/smartctl, you can inspect Percentage Used. "Percentage Used: Contains a vendor specific estimate of the percentage of life used for the Endurance Group based on the actual usage and the manufacturer’s prediction of NVM life. A value of 100 indicates that the estimated endurance of the NVM in the Endurance Group has been consumed, but may not indicate an NVM failure. The value is allowed to exceed 100." for SATA/SAS SSDs, there is "Media_Wearout_Indicator" which hasn't been a particularly reliable indicator in my experience.
  - Dalewyn 3y ago
    >if you get the SMART data with smartmontools/smartctl, you can inspect Percentage Used. CrystalDiskInfo[1] can be used for this purpose over on Windows. Some vendor-provided utilities like Samsung Magician will also provide this data with appropriate drives. [1]: https://crystalmark.info/en/software/crystaldiskinfo/ https://crystalmark.info/en/software/crystaldiskinfo/
    - justsomehnguy 3y ago
      > CrystalDiskInfo[1] can be used for this purpose over on Windows Only if you need a fancy GUI or guide some non-tech person to read the values to you over the phone/IM. Otherwise just use win32 port of smartmontools.
- magnetic 3y ago
  My SSDs show SMART attributes, which can be used as a rough indicator of health, but really the only strategy I've found to work well for my peace of mind is to use redundancy. Concretely, I use ZFS with a zpool with 2 SSDs in a mirror configuration. When one dies, even if it's sudden, I can just swap it out for another one and that's it. My vulnerability window starts when the first SSD fails and closes when the mirror is rebuilt. If something bad happens to the other SSD during that time, I'm toast and I have to start restoring from backup.
markhahn 3y ago
little hard to understand why it's worth explaining the details if you're going to gloss over the issue of endurance and erase cycle limits. if you do very little writing, you have nothing to worry about with SSD endurance. just read-disturb. do you do very little writing?
Eisenstein 3y ago
Something useful to know that wasn't mentioned: SSDs will corrupt data if sitting for extended periods without being powered on and thus should never be used for cold storage. "There are considerations which should be made if you are planning on shutting down an SSD based system for an extended period. The JEDEC spec for Enterprise SSD drives requires that the drives retain data for a minimum of 3 months at 40C. This means that after 3 months of a system being powered off in an environment that is at 40C or less, there is a potential of data loss and/or drive failures. This power off time limitation is due to the physical characteristics of Flash SSD media's gradual loss of electrical charge over an extended power down period. There is a potential for data loss and/or flash cell characteristic shift, leading to drive failure." * https://www.ibm.com/support/pages/potential-ssd-data-loss-after-extended-shutdown https://www.ibm.com/support/pages/potential-ssd-data-loss-af...
- robotnikman 3y ago
  Makes me wonder if those external SSD drives made by Samsung and WD/Sandisk take that into account and use different flash memory with better longevity without power.
  - sgerenser 3y ago
    Unlikely in the case of Sandisk, which is known for a recent bout of extremely unreliable external SSDs: https://arstechnica.com/gadgets/2023/08/sandisk-extreme-ssds-are-worthless-multiple-lawsuits-against-wd-say/ https://arstechnica.com/gadgets/2023/08/sandisk-extreme-ssds...
  - wmf 3y ago
    If anything, I would expect consumer SSDs to use the cheapest/lowest grade of flash available.
    - wtallis 3y ago
      Small USB flash drives get the worst NAND, followed by memory cards, then consumer SSDs, then enterprise SSDs. "Portable SSDs" that are physically much larger than USB thumb drives usually contain standard consumer SSDs in mSATA or M.2 form factor, plus a bridge chip.
MPSimmons 3y ago
11 years ago, I did a 3 hour 'Introduction to Solid State Storage' at LOPSA-East 2013 that also covered how spinning disks worked, if anyone is interested. https://www.youtube.com/watch?v=G3wf1HMr6b0 https://www.youtube.com/watch?v=G3wf1HMr6b0 The SSD content starts at around 1 hour in: https://youtu.be/G3wf1HMr6b0?si=5kdNeLGafrrU6Gmy&t=3573 https://youtu.be/G3wf1HMr6b0?si=5kdNeLGafrrU6Gmy&t=3573
creatonez 3y ago
There is a very good 5-part explanation from Branch Education: https://www.youtube.com/playlist?list=PL6rx9p3tbsMuk0jnC-dBdwb32Z1g7mD0j https://www.youtube.com/playlist?list=PL6rx9p3tbsMuk0jnC-dBd...
deleted 3y ago
[deleted]

207 points by fagnerbrack 3y ago

Two wonderful papers that are relevant: 1) https://pages.cs.wisc.edu/~jhe/eurosys17-he.pdf https://pages.cs.wisc.edu/~jhe/eurosys17-he.pdf 2) https://www.usenix.org/system/files/hotstorage19-paper-wu-kan.pdf https://www.usenix.org/system/files/hotstorage19-paper-wu-ka...

dang 3y ago

Discussed at the time: Everything I Know About SSDs - https://news.ycombinator.com/item?id=22054600 https://news.ycombinator.com/item?id=22054600 - Jan 2020 (185 comments)

eatonphil 3y ago

You might also be interested in this AMA we held on r/databasedevelopment with two NVMe developers from Samsung. https://www.reddit.com/r/databasedevelopment/comments/1afpezz/samsung_nvme_developers_ama/ https://www.reddit.com/r/databasedevelopment/comments/1afpez...

thadt 3y ago
Thanks for hosting (and posting it here). I was reading the "What Modern NVMe Storage Can Do..." paper just yesterday, and this was a great followup.

jmbwell 3y ago

I always thought 'flash' was a holdover from early reprogrammable ROM technology, non-volatile memory that you could erase by literally flashing a literal flashbulb over a little window on the chip. I would've sworn in a court of law that I recall this being called "flashable" memory, that erasing it was called "flashing" it, and reprogramming it in general was called "reflashing" in a sort of synecdoche. And I'd have assumed that this was the fundamental origin of what became _electronically_ erasable ROM (EEPROM), which led to all the various NVRAM technologies we have now, with "flashing" sticking as the term for reprogramming it, even after you could do it electronically. It looks like the story these days is that someone at Toshiba thought up the name out of the blue. I'm skeptical!

sgerenser 3y ago
I never heard of using a literal flash to reprogram EPROMs, but this wikipedia entry[1] makes your story for the origin of the term “Flashing” seem likely EPROMs are easily recognizable by the transparent fused quartz (or on later models resin) window on the top of the package, through which the silicon chip is visible, and which permits exposure to ultraviolet light during erasing. [1] https://en.m.wikipedia.org/wiki/EPROM https://en.m.wikipedia.org/wiki/EPROM
- jmbwell 3y ago
  It does look like all the references I can find point to engineers at Toshiba in 1980 coining the name, although Google ngrams shows some references to "flash EPROM" prior to 1980, so I can't help but wonder if the idea existed at least in some form prior to 1980
  - monocasa 3y ago
    Looking into it, those references look miscategorized. It's some Zambian national report that's talking about 0.18um processes (aka, 180nm) in the same paragraph, which wouldn't have come out until the very late 90s.
- radicalbyte 3y ago
  These things were used in a number of computers from the early 90s, I have a few lying around for my Amiga.
- hex4def6 3y ago
  Heard from a greybeard that they had a demo night in university (this was probably early 80s), and one of the demos was was some sort of path-finding rover robot. Of course, it ended up being one of those projects that had show stopping bugs up until a few hours before showtime. During demo night, it was a big hit, until a stray camera flash got (un)lucky and wiped the microcontroller's EEPROM... Feels a little apocryphal (I'd assume most flashes have / had UV filters?)

password4321 3y ago

I've heard SSDs are more likely to "fail fast". Can anyone recommended utilities that monitor and warn before SSD failures?

caseyf 3y ago
For NVMe, if you get the SMART data with smartmontools/smartctl, you can inspect Percentage Used. "Percentage Used: Contains a vendor specific estimate of the percentage of life used for the Endurance Group based on the actual usage and the manufacturer’s prediction of NVM life. A value of 100 indicates that the estimated endurance of the NVM in the Endurance Group has been consumed, but may not indicate an NVM failure. The value is allowed to exceed 100." for SATA/SAS SSDs, there is "Media_Wearout_Indicator" which hasn't been a particularly reliable indicator in my experience.
- Dalewyn 3y ago
  >if you get the SMART data with smartmontools/smartctl, you can inspect Percentage Used. CrystalDiskInfo[1] can be used for this purpose over on Windows. Some vendor-provided utilities like Samsung Magician will also provide this data with appropriate drives. [1]: https://crystalmark.info/en/software/crystaldiskinfo/ https://crystalmark.info/en/software/crystaldiskinfo/
  - justsomehnguy 3y ago
    > CrystalDiskInfo[1] can be used for this purpose over on Windows Only if you need a fancy GUI or guide some non-tech person to read the values to you over the phone/IM. Otherwise just use win32 port of smartmontools.
magnetic 3y ago
My SSDs show SMART attributes, which can be used as a rough indicator of health, but really the only strategy I've found to work well for my peace of mind is to use redundancy. Concretely, I use ZFS with a zpool with 2 SSDs in a mirror configuration. When one dies, even if it's sudden, I can just swap it out for another one and that's it. My vulnerability window starts when the first SSD fails and closes when the mirror is rebuilt. If something bad happens to the other SSD during that time, I'm toast and I have to start restoring from backup.

markhahn 3y ago

little hard to understand why it's worth explaining the details if you're going to gloss over the issue of endurance and erase cycle limits. if you do very little writing, you have nothing to worry about with SSD endurance. just read-disturb. do you do very little writing?

Eisenstein 3y ago

Something useful to know that wasn't mentioned: SSDs will corrupt data if sitting for extended periods without being powered on and thus should never be used for cold storage. "There are considerations which should be made if you are planning on shutting down an SSD based system for an extended period. The JEDEC spec for Enterprise SSD drives requires that the drives retain data for a minimum of 3 months at 40C. This means that after 3 months of a system being powered off in an environment that is at 40C or less, there is a potential of data loss and/or drive failures. This power off time limitation is due to the physical characteristics of Flash SSD media's gradual loss of electrical charge over an extended power down period. There is a potential for data loss and/or flash cell characteristic shift, leading to drive failure." * https://www.ibm.com/support/pages/potential-ssd-data-loss-after-extended-shutdown https://www.ibm.com/support/pages/potential-ssd-data-loss-af...

robotnikman 3y ago
Makes me wonder if those external SSD drives made by Samsung and WD/Sandisk take that into account and use different flash memory with better longevity without power.
- sgerenser 3y ago
  Unlikely in the case of Sandisk, which is known for a recent bout of extremely unreliable external SSDs: https://arstechnica.com/gadgets/2023/08/sandisk-extreme-ssds-are-worthless-multiple-lawsuits-against-wd-say/ https://arstechnica.com/gadgets/2023/08/sandisk-extreme-ssds...
- wmf 3y ago
  If anything, I would expect consumer SSDs to use the cheapest/lowest grade of flash available.
  - wtallis 3y ago
    Small USB flash drives get the worst NAND, followed by memory cards, then consumer SSDs, then enterprise SSDs. "Portable SSDs" that are physically much larger than USB thumb drives usually contain standard consumer SSDs in mSATA or M.2 form factor, plus a bridge chip.

MPSimmons 3y ago

11 years ago, I did a 3 hour 'Introduction to Solid State Storage' at LOPSA-East 2013 that also covered how spinning disks worked, if anyone is interested. https://www.youtube.com/watch?v=G3wf1HMr6b0 https://www.youtube.com/watch?v=G3wf1HMr6b0 The SSD content starts at around 1 hour in: https://youtu.be/G3wf1HMr6b0?si=5kdNeLGafrrU6Gmy&t=3573 https://youtu.be/G3wf1HMr6b0?si=5kdNeLGafrrU6Gmy&t=3573

creatonez 3y ago

There is a very good 5-part explanation from Branch Education: https://www.youtube.com/playlist?list=PL6rx9p3tbsMuk0jnC-dBdwb32Z1g7mD0j https://www.youtube.com/playlist?list=PL6rx9p3tbsMuk0jnC-dBd...

deleted 3y ago

[deleted]