8 ms·

The Unwritten Contract of Solid State Drives (2017)

110 points by jitl 5y ago

bifrost 5y ago
Its weird they didn't test with ZFS, UFS/FFS or ReiserFS.
- kbenson 5y ago
  > Its weird they didn't test with ZFS Maybe because of the numerous ways if can be configured that might affect results. ZFS is not as simple as Ext4 or XFS in configuration, and it might have just made it harder to get useful information from the results. At the same time, someone can probably use the results here to make some educated guesses as to how ZFS can/should be configured for SSDs based on this and some knowledge of SSDs. > UFS/FFS The purpose isn't to test filesystems, it's to figure out how to best leverage SSDs specific performance and design characteristics for both immediate and long term performance. > or ReiserFS Uh, is ReiserFS still actively developed? I used to follow it quite actively, but I admit I sort of lost interest after it seemed to stall after Han Reiser's murder conviction.
  - bifrost 5y ago
    RE: ZFS - Testing with modern filesystems should be a key point for benchmarking and "getting the most out of a device". Because ZFS contains both logical volume management and a variety of other safety/speed controls it really should've been included. RE: UFS/FFS - These perform quite well with SSDs and in various configurations will outperform ext4/etc in terms of sheer write/read performance and also data safety.
deleted 5y ago
[deleted]
sideshowb 5y ago
I know journals differ in their guidance, but shouldn't an abstract say something about the findings? That one reads more like the back cover of a paperback... (No offense intended to the authors for this fine piece of work, nor to paperback writers for their ability to create suspense. Just a gripe about metadata!)
- teekert 5y ago
  Indeed! And where is HN when you need it ;) I for one will be refreshing this page over the course of this day hoping for an unsung hero to come along, and I shall be upvoting.
  - sideshowb 5y ago
    teekert and sideshowb sat idly on the couch, uninspired to scroll down to the discussion section. Would a tall, handsome hero show up to lift them to new heights of summarization? Only time would tell...
    - dvfjsdhgfv 5y ago
      I doubt they would show up this time as the study has no single conclusion, just a number of observations, nothing really striking.
  - willis936 5y ago
    There are links to eReader and PDF in bright buttons at the top of the page. https://dl.acm.org/doi/epdf/10.1145/3064176.3064187 https://dl.acm.org/doi/epdf/10.1145/3064176.3064187 https://dl.acm.org/doi/pdf/10.1145/3064176.3064187 https://dl.acm.org/doi/pdf/10.1145/3064176.3064187
    - sideshowb 5y ago
      The abstract on those is the same in that it does not summarize findings.
- KineticLensman 5y ago
  > shouldn't an abstract say something about the findings? One counter-example is in security / defence work where the existence of a project is unclassified but the details are protected in some way. Not reporting the findings allows the paper to be indexed in libraries / registries without 'contaminating' the library indexes with classified information. The other example (rightly or wrongly) is where a publisher wants you to be able to find via bibliographic search tools a paper hidden behind a paywall but still wants you to pay to access the paper. If the abstract revealed too much, it defeats the paywall, from the publisher's perspective.
deleted 5y ago
[deleted]
nabla9 5y ago
Unwritten 5 rules are: Request Scale: SSD clients should issue large data requests or multiple concurrent requests. A small request scale leads to low resource utilization and reduces immediate and sustainable performance Locality rule: SSD clients should access with locality. Workloads without locality can incur a poor immediate performance because frequent cache misses lead to many translation-related reads and writes.Poor locality also impacts sustainable performance because data movement during garbage collection and wear-leveling requires translations and mapping updates Aligned Sequentiality rule: clients of SSDs with hybrid FTLs should start writing at the aligned beginning of a block boundary and write sequentially. This rule does not affect immediate performance since the conversion happens later, but violating this rule degrades sustainable performance because of costly data movement during the delayed conversion. Grouping By Death Time rule: The death time of a page is the time the page is discarded or overwritten by the host. If a block has data with different death times, then there is a time window between the first and last page invalidations within which both live and dead data reside in the block. There are two practical ways to achieve this: 1) data with similar death times are gathered in the write sequence 2) placing death groups to different logical segments isolates them physically Uniform Lifetime rule: clients of SSDs should create data with similar lifetimes. Lack of lifetime uniformity does not directly impact immediate performance,but impacts sustainable performance as it necessitates wear-leveling and leads to loss of capacity.
- povik 5y ago
  Regarding the FTL acronym: "To hide the complexity of SSD internals, the controller usually contains a piece of software called an FTL (Flash Translation Layer); the FTL provides the host with a simple block interface and manages all the operations on the flash chips." ... "A hybrid FTL uses page-level mappings for new data and converts them to block-level mappings when it runs out of mapping cache."
- wtallis 5y ago
  The concerns in the paper motivating the locality rule really only apply to low-end consumer SSDs. Enterprise/datacenter SSDs and high-end consumer SSDs all carry plenty of DRAM for the FTL, so they don't suffer as much penalty for poor locality and it's really only worth drawing a distinction between sequential and non-sequential access. I also don't think it's common for SSDs to use the kind of hybrid page/block mapping FTL that the Aligned Sequentiality rule is about. These days, if you want a SSD that cuts down on DRAM costs, you want the NVMe Zoned Namespaces feature. The problems that the Uniform Lifetime rule and Grouping By Death Time rule are concerned with can be handled explicitly with features like NVMe's Streams directives, or Zoned Namespaces. Overall, this article doesn't seem to have aged particularly well, and I'm not sure it was that useful and relevant even in 2017.
bullen 5y ago
I just bought 30x new 64GB X25-E 50nm Intel SSDs from 2011 from ebay, they are SLC and should manage 100.000 writes per bit and I don't think we'll ever see that kind of quality again. I'm going to use EXT4 (with type small because my custom distributed database writes many small JSON files) and hope for the best on my Atom 8/16-core machines that have 12 SATA ports. I wont use raid but instead plan to mount these "manually". Would love some feedback! :S
- wtallis 5y ago
  If you want to mess around with small fast drives with high endurance relative to their capacity, you should consider the less outdated Intel Optane Memory cache drives, which are also far faster than ancient 3Gbps SATA drives.
  - bullen 5y ago
    I saw those on ebay but the connector was weird also I don't need more speed, these machines will be behind 2x separate home 1Gb/s (up and down) fiber... Another thing I had to consider was power since I plan to back these up with lead-acid batteries for power outages. These drives draw 2W when active... so that is 18W for 8x (512GB) DB drives and one OS drive! Compared to the 25/32W of the 8/16-core motherboards that's alot... but mechanical drives are insane on electricity, up to 10W per drive!!! But good point, do you know what connector tese use?
    - wtallis 5y ago
      The Optane Memory drives use a M.2 connector, keyed for carrying two lanes of PCI Express 3.0, over which they use the NVMe protocol. If you're concerned about battery life, you really shouldn't be using lots of small drives, especially when the controllers are so ancient. You'll get at least 5x the performance per Watt from a single newer flash-based SSD and much better idle power behavior. Consumer SSDs also offer much more in the way of low-power idle states than enterprise SSDs. At the extreme, you can get a $75 512GB drive that provides significantly more performance than 8x X25-Es in aggregate, while never reaching 3W power draw under load and idling at under 10mW.
jokoon 5y ago
I have old pictures I took, they were on a SD card. For some reason a few of those files are 0 bytes, meaning they probably went corrupt at some point. It seems that it also happens on my SSD. For some other reason, windows explorer seems to crash or hang when I browse folder with a lot of images. SSD are known to have a short life expectancy. I wonder if there is a market for very long life expectancy for HDDs. I mean any type of data storage that can last for 10 years without failure would have value. Of course magnetic tape systems exist but I would guess that very durable HDD would still be better.
- calvinmorrison 5y ago
  CDs are probably the sweet spot
  - timw4mail 5y ago
    For cheaply storing data, sure. The dye on burnable discs tends to degrade over time, though, making them poor for archival purposes.
    - dragontamer 5y ago
      Everything degrades over time. An archival quality DVD has dyes that resists the bitrot. Archival quality magnetic mediums (HDDs or Tape, or floppies) also resist the bitrot... but they're not immune. The only way to fully avoid bitrot is to constantly read-and-rewrite the data. You need to read the data before it degrades: that could be once a year, or maybe every few months. But cold-storage is just a devilishly hard problem. Its probably best to just not deal with that at all. Just get a NAS and read/rewrite the data automatically (aka: "scrub" your data in ZFS terms)
  - dragontamer 5y ago
    they WERE the sweet spot at one point (at least, DVDs were), but they've fallen behind and are no longer worth it. All the issues with hard drives (single "read" mechanism, limiting bandwidth) are doubly so when the medium exists outside of the read-mechanism and multiple manufacturers need to try to stay compatible with each other. It seems difficult to improve the medium independently of the read-mechanism. I don't really know how the tape-drive industry manages to keep up. Tape just has some huge fundamental advantages in surface area I guess... but Tape also only promises read compatibility for 2-generations. So the whole industry manages to move forward on a decent cadence. --------- SSDs are now the right size and price for typical end-user devices. But HDDs are still the king of capacity for all datasets below 50TBs, while Tape reigns supreme at the highest end in price per TB.

110 points by jitl 5y ago

bifrost 5y ago

Its weird they didn't test with ZFS, UFS/FFS or ReiserFS.

kbenson 5y ago
> Its weird they didn't test with ZFS Maybe because of the numerous ways if can be configured that might affect results. ZFS is not as simple as Ext4 or XFS in configuration, and it might have just made it harder to get useful information from the results. At the same time, someone can probably use the results here to make some educated guesses as to how ZFS can/should be configured for SSDs based on this and some knowledge of SSDs. > UFS/FFS The purpose isn't to test filesystems, it's to figure out how to best leverage SSDs specific performance and design characteristics for both immediate and long term performance. > or ReiserFS Uh, is ReiserFS still actively developed? I used to follow it quite actively, but I admit I sort of lost interest after it seemed to stall after Han Reiser's murder conviction.
- bifrost 5y ago
  RE: ZFS - Testing with modern filesystems should be a key point for benchmarking and "getting the most out of a device". Because ZFS contains both logical volume management and a variety of other safety/speed controls it really should've been included. RE: UFS/FFS - These perform quite well with SSDs and in various configurations will outperform ext4/etc in terms of sheer write/read performance and also data safety.

deleted 5y ago

[deleted]

sideshowb 5y ago

I know journals differ in their guidance, but shouldn't an abstract say something about the findings? That one reads more like the back cover of a paperback... (No offense intended to the authors for this fine piece of work, nor to paperback writers for their ability to create suspense. Just a gripe about metadata!)

teekert 5y ago
Indeed! And where is HN when you need it ;) I for one will be refreshing this page over the course of this day hoping for an unsung hero to come along, and I shall be upvoting.
- sideshowb 5y ago
  teekert and sideshowb sat idly on the couch, uninspired to scroll down to the discussion section. Would a tall, handsome hero show up to lift them to new heights of summarization? Only time would tell...
  - dvfjsdhgfv 5y ago
    I doubt they would show up this time as the study has no single conclusion, just a number of observations, nothing really striking.
- willis936 5y ago
  There are links to eReader and PDF in bright buttons at the top of the page. https://dl.acm.org/doi/epdf/10.1145/3064176.3064187 https://dl.acm.org/doi/epdf/10.1145/3064176.3064187 https://dl.acm.org/doi/pdf/10.1145/3064176.3064187 https://dl.acm.org/doi/pdf/10.1145/3064176.3064187
  - sideshowb 5y ago
    The abstract on those is the same in that it does not summarize findings.
KineticLensman 5y ago
> shouldn't an abstract say something about the findings? One counter-example is in security / defence work where the existence of a project is unclassified but the details are protected in some way. Not reporting the findings allows the paper to be indexed in libraries / registries without 'contaminating' the library indexes with classified information. The other example (rightly or wrongly) is where a publisher wants you to be able to find via bibliographic search tools a paper hidden behind a paywall but still wants you to pay to access the paper. If the abstract revealed too much, it defeats the paywall, from the publisher's perspective.

deleted 5y ago

[deleted]

nabla9 5y ago

Unwritten 5 rules are: Request Scale: SSD clients should issue large data requests or multiple concurrent requests. A small request scale leads to low resource utilization and reduces immediate and sustainable performance Locality rule: SSD clients should access with locality. Workloads without locality can incur a poor immediate performance because frequent cache misses lead to many translation-related reads and writes.Poor locality also impacts sustainable performance because data movement during garbage collection and wear-leveling requires translations and mapping updates Aligned Sequentiality rule: clients of SSDs with hybrid FTLs should start writing at the aligned beginning of a block boundary and write sequentially. This rule does not affect immediate performance since the conversion happens later, but violating this rule degrades sustainable performance because of costly data movement during the delayed conversion. Grouping By Death Time rule: The death time of a page is the time the page is discarded or overwritten by the host. If a block has data with different death times, then there is a time window between the first and last page invalidations within which both live and dead data reside in the block. There are two practical ways to achieve this: 1) data with similar death times are gathered in the write sequence 2) placing death groups to different logical segments isolates them physically Uniform Lifetime rule: clients of SSDs should create data with similar lifetimes. Lack of lifetime uniformity does not directly impact immediate performance,but impacts sustainable performance as it necessitates wear-leveling and leads to loss of capacity.

povik 5y ago
Regarding the FTL acronym: "To hide the complexity of SSD internals, the controller usually contains a piece of software called an FTL (Flash Translation Layer); the FTL provides the host with a simple block interface and manages all the operations on the flash chips." ... "A hybrid FTL uses page-level mappings for new data and converts them to block-level mappings when it runs out of mapping cache."
wtallis 5y ago
The concerns in the paper motivating the locality rule really only apply to low-end consumer SSDs. Enterprise/datacenter SSDs and high-end consumer SSDs all carry plenty of DRAM for the FTL, so they don't suffer as much penalty for poor locality and it's really only worth drawing a distinction between sequential and non-sequential access. I also don't think it's common for SSDs to use the kind of hybrid page/block mapping FTL that the Aligned Sequentiality rule is about. These days, if you want a SSD that cuts down on DRAM costs, you want the NVMe Zoned Namespaces feature. The problems that the Uniform Lifetime rule and Grouping By Death Time rule are concerned with can be handled explicitly with features like NVMe's Streams directives, or Zoned Namespaces. Overall, this article doesn't seem to have aged particularly well, and I'm not sure it was that useful and relevant even in 2017.

bullen 5y ago

I just bought 30x new 64GB X25-E 50nm Intel SSDs from 2011 from ebay, they are SLC and should manage 100.000 writes per bit and I don't think we'll ever see that kind of quality again. I'm going to use EXT4 (with type small because my custom distributed database writes many small JSON files) and hope for the best on my Atom 8/16-core machines that have 12 SATA ports. I wont use raid but instead plan to mount these "manually". Would love some feedback! :S

wtallis 5y ago
If you want to mess around with small fast drives with high endurance relative to their capacity, you should consider the less outdated Intel Optane Memory cache drives, which are also far faster than ancient 3Gbps SATA drives.
- bullen 5y ago
  I saw those on ebay but the connector was weird also I don't need more speed, these machines will be behind 2x separate home 1Gb/s (up and down) fiber... Another thing I had to consider was power since I plan to back these up with lead-acid batteries for power outages. These drives draw 2W when active... so that is 18W for 8x (512GB) DB drives and one OS drive! Compared to the 25/32W of the 8/16-core motherboards that's alot... but mechanical drives are insane on electricity, up to 10W per drive!!! But good point, do you know what connector tese use?
  - wtallis 5y ago
    The Optane Memory drives use a M.2 connector, keyed for carrying two lanes of PCI Express 3.0, over which they use the NVMe protocol. If you're concerned about battery life, you really shouldn't be using lots of small drives, especially when the controllers are so ancient. You'll get at least 5x the performance per Watt from a single newer flash-based SSD and much better idle power behavior. Consumer SSDs also offer much more in the way of low-power idle states than enterprise SSDs. At the extreme, you can get a $75 512GB drive that provides significantly more performance than 8x X25-Es in aggregate, while never reaching 3W power draw under load and idling at under 10mW.

jokoon 5y ago

I have old pictures I took, they were on a SD card. For some reason a few of those files are 0 bytes, meaning they probably went corrupt at some point. It seems that it also happens on my SSD. For some other reason, windows explorer seems to crash or hang when I browse folder with a lot of images. SSD are known to have a short life expectancy. I wonder if there is a market for very long life expectancy for HDDs. I mean any type of data storage that can last for 10 years without failure would have value. Of course magnetic tape systems exist but I would guess that very durable HDD would still be better.

calvinmorrison 5y ago
CDs are probably the sweet spot
- timw4mail 5y ago
  For cheaply storing data, sure. The dye on burnable discs tends to degrade over time, though, making them poor for archival purposes.
  - dragontamer 5y ago
    Everything degrades over time. An archival quality DVD has dyes that resists the bitrot. Archival quality magnetic mediums (HDDs or Tape, or floppies) also resist the bitrot... but they're not immune. The only way to fully avoid bitrot is to constantly read-and-rewrite the data. You need to read the data before it degrades: that could be once a year, or maybe every few months. But cold-storage is just a devilishly hard problem. Its probably best to just not deal with that at all. Just get a NAS and read/rewrite the data automatically (aka: "scrub" your data in ZFS terms)
- dragontamer 5y ago
  they WERE the sweet spot at one point (at least, DVDs were), but they've fallen behind and are no longer worth it. All the issues with hard drives (single "read" mechanism, limiting bandwidth) are doubly so when the medium exists outside of the read-mechanism and multiple manufacturers need to try to stay compatible with each other. It seems difficult to improve the medium independently of the read-mechanism. I don't really know how the tape-drive industry manages to keep up. Tape just has some huge fundamental advantages in surface area I guess... but Tape also only promises read compatibility for 2-generations. So the whole industry manages to move forward on a decent cadence. --------- SSDs are now the right size and price for typical end-user devices. But HDDs are still the king of capacity for all datasets below 50TBs, while Tape reigns supreme at the highest end in price per TB.