3 ms·

Probably the biggest ground-up rethink of SSDs that I've seen is the Samsung SSDs with a key-value store mode. Like, think about it, a filesystem is really a d

by paulmd 3y ago

Probably the biggest ground-up rethink of SSDs that I've seen is the Samsung SSDs with a key-value store mode. Like, think about it, a filesystem is really a database, right? Copy-on-write even makes the "transactionality" explicit. And a lot of high-performance databases will go farther and skip the filesystem and treat the device as block storage... which it is. The filesystem is a leaky abstraction with filesystem blocks and flash pages and flash block erasure, etc. Well, what if the SSD was just an object store? In that model you slice away a bunch of layers of abstraction and just let the SSD worry about all that. SSD says it's committed? Alright then, guess it is. Obviously you are very much at the mercy of the SSD to implement ACID correctly however...

r1ch 3y ago
Optane was also a contender for the biggest fundamental shift - byte addressable persistent memory with insane endurance is simply in a different class to every other SSD technology out there. No need for a FTL. No need for a DRAM / SLC cache. Unfortunately no software really took advantage of it, OSes still provisioned it as 512/4096 byte sectors, etc. so in standard benchmarks it never really looked that compelling over a regular NVME drive. But if the access patterns align, nothing can come close. Truly a technology ahead of its time.
- MichaelZuo 3y ago
  It always made me surprised that Intel didn't market it better, such as forking linux and writing some custom additions that could allow the Optane drives to demonstrate real world 10x improvements.
  - paulmd 3y ago
    Intel already has Clear Linux actually. I don't think there's really a ton you can do at an OS level. ZFS L2ARC, system swap, etc, but a lot of the benefits would come from tuning DB configs/etc to suit the new hardware. If it's a fast SSD... ok are you the DBA? Similarly, PDIMM is really best utilized by a ground-up rethink of applications operating in a persistent context (think of like, javacard multicore 1TB or whatever). If you treat it as slow RAM it's gonna be slow RAM. The point would be building systems and applications that exploit the "writing the memory is now writing the disk" idiom. The ideal ecosytem for PDIMM isn't Linux at all, it's JVM, or LISP/Haskell. Or android I suppose lol.
  - PaulHoule 3y ago
    Optane was either the fastest SSD you ever had or the slowest RAM. Incorporating it into the memory hierarchy might not be a win if you are replacing fast RAM with slow Optane.
    - MichaelZuo 3y ago
      I assume for marketing figures, Optane would replace the SSD at a much higher cost.
throwawaylinux 3y ago
Key value FTLs have been around for about as long as FTLs. The logical block address -> NAND page forward map is just a key value store after all. It can potentially take one layer of indirection out of applications but you still have one there (the FTL). Or alternatively exposing the raw NAND and having the software manage it is another common way to go (indirection is in software now instead of hardware/firmware). When I worked on FTLs it was always a thing executives and product managers liked to talk about but didn't seem to provide incredible results and people generally didn't like using them because it was just harder / different to manage. Has that changed?
- anyfoo 3y ago
  > Key value FTLs have been around for about as long as FTLs. Is this similar to what IBM had since the 60s already on their hard disks (which they call "DASD"), namely "CKD" (count-key-data) storage?
evntdrvn 3y ago
fwiw Seagate also experimented with this for hard drives, it was pretty neat--ethernet-connected KV-addressed hard drives. I don't think it made it to an actual shipped product, but it did make it to "partner field testing phase".

by paulmd 3y ago

r1ch 3y ago

Optane was also a contender for the biggest fundamental shift - byte addressable persistent memory with insane endurance is simply in a different class to every other SSD technology out there. No need for a FTL. No need for a DRAM / SLC cache. Unfortunately no software really took advantage of it, OSes still provisioned it as 512/4096 byte sectors, etc. so in standard benchmarks it never really looked that compelling over a regular NVME drive. But if the access patterns align, nothing can come close. Truly a technology ahead of its time.

MichaelZuo 3y ago
It always made me surprised that Intel didn't market it better, such as forking linux and writing some custom additions that could allow the Optane drives to demonstrate real world 10x improvements.
- paulmd 3y ago
  Intel already has Clear Linux actually. I don't think there's really a ton you can do at an OS level. ZFS L2ARC, system swap, etc, but a lot of the benefits would come from tuning DB configs/etc to suit the new hardware. If it's a fast SSD... ok are you the DBA? Similarly, PDIMM is really best utilized by a ground-up rethink of applications operating in a persistent context (think of like, javacard multicore 1TB or whatever). If you treat it as slow RAM it's gonna be slow RAM. The point would be building systems and applications that exploit the "writing the memory is now writing the disk" idiom. The ideal ecosytem for PDIMM isn't Linux at all, it's JVM, or LISP/Haskell. Or android I suppose lol.
- PaulHoule 3y ago
  Optane was either the fastest SSD you ever had or the slowest RAM. Incorporating it into the memory hierarchy might not be a win if you are replacing fast RAM with slow Optane.
  - MichaelZuo 3y ago
    I assume for marketing figures, Optane would replace the SSD at a much higher cost.

throwawaylinux 3y ago

Key value FTLs have been around for about as long as FTLs. The logical block address -> NAND page forward map is just a key value store after all. It can potentially take one layer of indirection out of applications but you still have one there (the FTL). Or alternatively exposing the raw NAND and having the software manage it is another common way to go (indirection is in software now instead of hardware/firmware). When I worked on FTLs it was always a thing executives and product managers liked to talk about but didn't seem to provide incredible results and people generally didn't like using them because it was just harder / different to manage. Has that changed?

anyfoo 3y ago
> Key value FTLs have been around for about as long as FTLs. Is this similar to what IBM had since the 60s already on their hard disks (which they call "DASD"), namely "CKD" (count-key-data) storage?

evntdrvn 3y ago

fwiw Seagate also experimented with this for hard drives, it was pretty neat--ethernet-connected KV-addressed hard drives. I don't think it made it to an actual shipped product, but it did make it to "partner field testing phase".