4 ms·

SMR Hard Drives have very different rules about how you should access them vs conventional hard drives or SSDs. I wonder how much optimizing for SMR drives (Bi

by Dwedit 5mo ago

SMR Hard Drives have very different rules about how you should access them vs conventional hard drives or SSDs. I wonder how much optimizing for SMR drives (Big sequential writes) would also optimize for other drive types.

deleted 5mo ago
[deleted]
jauntywundrkind 5mo ago
The zoned-storage people (whom shingled folks were a subsect of) seemed pretty ok with the FDP (Flexible Data Placement - TP4146b) scheme that finally finally finally got hammered out for NVMe 2.1 (August 2024). It was also designed to satisfy the open-channel flash people as well. It's a fairly simple concept that lets you have some write-affinity, that lets you declare when writing that this write should be associated with other writes with the same FDP number, a tagging. I'm not fully convinced this really is as good as what the open channel flash people wanted. But drive manufacturers were never voluntarily going to give up really complex Flash Translation Layers. They all want to be value add, have their expensive fancy controllers keeping the market from commoditizing to just using NAND directly. But FDP does show some very real promises, can have huge read-write-affinity bonuses! I note that SSDFS filesystem is still out there being improved and maintained, for a file system that tries to take advantage of this all. I'm not sure if it's made the jump to using FDP or is still older much more ornery & never quite loved ZNS specification. I'd love to give it a try but FDP and ZNS drives are not easy to get ahold of, require asking very nicely, and when I last checked required purchasing very expensive fancy enterprise SSD that cost a ton but which had pretty so so performance figures. That was a couple years ago now. https://www.phoronix.com/news/Linux-SSDFS-NVMe-ZNS-SSDs https://www.phoronix.com/news/Linux-SSDFS-NVMe-ZNS-SSDs https://news.ycombinator.com/item?id=34939248 https://news.ycombinator.com/item?id=34939248 The paper here is wonderful & beautiful. FDP should make this kind of thing so so so much easierz should remove so many of the downsides of drive usage mentioned here. If only it were available. I'd really love it if driver reviewers would look at and comment on the feature matrix drives have, and comment on FDP, but generally, it feels like there's no ask, little pull and thus no push, for an obvious and basically free to implement zero cost improvement that makes everything vastly better. Alas. Can't wait. Hopefully drive prices are better by 2031 & FDP is finally available. Fingers crossed.
- geertj 5mo ago
  Speaking of zoned SSDs (ZNS or FDP), are any of these available today without having to ‘call sales’? I wanted to experiment with this maybe 2 years back and there was nothing.
  - jauntywundrkind 5mo ago
    With apologies for outing your bio here so pointedly, > Storage nerd @ Google Vendors, are you listening? (And the software would/could be so much better... If this were available to play with)
    - geertj 5mo ago
      To be clear, I was asking from a personal perspective, unrelated to my role at Google.
UltraSane 5mo ago
SMR drives shouldn't be used with write heavy databases. Or really any write heavy workload. They are best used for storing lots of read only.

by Dwedit 5mo ago

deleted 5mo ago

[deleted]

jauntywundrkind 5mo ago

The zoned-storage people (whom shingled folks were a subsect of) seemed pretty ok with the FDP (Flexible Data Placement - TP4146b) scheme that finally finally finally got hammered out for NVMe 2.1 (August 2024). It was also designed to satisfy the open-channel flash people as well. It's a fairly simple concept that lets you have some write-affinity, that lets you declare when writing that this write should be associated with other writes with the same FDP number, a tagging. I'm not fully convinced this really is as good as what the open channel flash people wanted. But drive manufacturers were never voluntarily going to give up really complex Flash Translation Layers. They all want to be value add, have their expensive fancy controllers keeping the market from commoditizing to just using NAND directly. But FDP does show some very real promises, can have huge read-write-affinity bonuses! I note that SSDFS filesystem is still out there being improved and maintained, for a file system that tries to take advantage of this all. I'm not sure if it's made the jump to using FDP or is still older much more ornery & never quite loved ZNS specification. I'd love to give it a try but FDP and ZNS drives are not easy to get ahold of, require asking very nicely, and when I last checked required purchasing very expensive fancy enterprise SSD that cost a ton but which had pretty so so performance figures. That was a couple years ago now. https://www.phoronix.com/news/Linux-SSDFS-NVMe-ZNS-SSDs https://www.phoronix.com/news/Linux-SSDFS-NVMe-ZNS-SSDs https://news.ycombinator.com/item?id=34939248 https://news.ycombinator.com/item?id=34939248 The paper here is wonderful & beautiful. FDP should make this kind of thing so so so much easierz should remove so many of the downsides of drive usage mentioned here. If only it were available. I'd really love it if driver reviewers would look at and comment on the feature matrix drives have, and comment on FDP, but generally, it feels like there's no ask, little pull and thus no push, for an obvious and basically free to implement zero cost improvement that makes everything vastly better. Alas. Can't wait. Hopefully drive prices are better by 2031 & FDP is finally available. Fingers crossed.

geertj 5mo ago
Speaking of zoned SSDs (ZNS or FDP), are any of these available today without having to ‘call sales’? I wanted to experiment with this maybe 2 years back and there was nothing.
- jauntywundrkind 5mo ago
  With apologies for outing your bio here so pointedly, > Storage nerd @ Google Vendors, are you listening? (And the software would/could be so much better... If this were available to play with)
  - geertj 5mo ago
    To be clear, I was asking from a personal perspective, unrelated to my role at Google.

UltraSane 5mo ago

SMR drives shouldn't be used with write heavy databases. Or really any write heavy workload. They are best used for storing lots of read only.