4 ms·

Yeah optane seemed like it had so much potential -- with the right abstractions, it neatly dealt with the eternal problem of the separation between RAM and disk

Yeah optane seemed like it had so much potential -- with the right abstractions, it neatly dealt with the eternal problem of the separation between RAM and disk. Much of DB and storage system engineering comes down to decisions around when to persist what, and optane was very promising in allowing for new architectures that had much better performance at lower complexity. Alas, optane's dead now. I do know people actively working on resurrecting a lot of pmem work on low-latency flash, however, and it seems like this is one area with a low of momentum behind it.

paulmd 3y ago
I'm utterly unqualified to discuss using it in practice but it always seemed like PMEM would have required a fundamental idiom shift away from the DRAM concept. If you treat it like memory it's always going to be slow memory. If you treat it like storage, it can be very fast storage. If that makes sense. Software that "treats storage like memory" would end up looking a lot like javacard imo. Or something like Samsung's in-memory key-value database stuff. But it wouldn't really look like a linux kernel allocating memory inside an all-pmem partition. Difference, in my head, being that essentially the filesystem is an unnecessary layer/abstraction in the middle. You want something that either looks like a garbage-collected runtime, or an in-memory database that vacuums, or LISP collections of object-trees, etc. There is no point to having a separation between memory and disk anymore, that is the point - "resource allocation is disk persistence" so to speak. It would have required a big-bang rewrite/second-system that is just not possible with the dominance of the existing RAM/storage dichotomy. Or at least a couple killer apps from large vendors/etc that really outperformed what was possible with RAM.
- dvas 3y ago
  Some support has been added to the Linux kernel like DAX [0] to avoid extra copy operations [0]. SNIA has some interesting talks regarding using non volatile memory and its applications (DBs). Where using pmem and DAX to directly to store logging operations [1]. [0] What is Direct-Access (DAX)? https://kb.pmem.io/faq/100000008-What-is-DAX/ https://kb.pmem.io/faq/100000008-What-is-DAX/ [1] SDC2020: How can Persistent Memory Make Databases Faster, and How Could we go Ahead? https://www.youtube.com/watch?v=rTgITrVhpQM https://www.youtube.com/watch?v=rTgITrVhpQM
  - deleted 3y ago
    [deleted]
- gavinray 3y ago
  Pavlo has some good papers on architecting DB's around PMEM "How to Build a Non-Volatile Memory Database Management System" https://db.cs.cmu.edu/papers/2017/p1753-arulraj.pdf https://db.cs.cmu.edu/papers/2017/p1753-arulraj.pdf
  - dvas 3y ago
    Under 3.1 Allocator Interface, there is a great point about reducing wear levelling by rebalancing the B+ tree operations to reduce the number of writes thus extending the lifespan of the hardware. Not much of a concern on the small scale, however in large deployments even single-digit improvements will make a difference. Sometimes it becomes useful to appreciate the hardware and what it is doing, reminds me of the quote. * You don't have to be an engineer to be a racing driver, but you do have to have Mechanical Sympathy *. Jackie Stewart

paulmd 3y ago

I'm utterly unqualified to discuss using it in practice but it always seemed like PMEM would have required a fundamental idiom shift away from the DRAM concept. If you treat it like memory it's always going to be slow memory. If you treat it like storage, it can be very fast storage. If that makes sense. Software that "treats storage like memory" would end up looking a lot like javacard imo. Or something like Samsung's in-memory key-value database stuff. But it wouldn't really look like a linux kernel allocating memory inside an all-pmem partition. Difference, in my head, being that essentially the filesystem is an unnecessary layer/abstraction in the middle. You want something that either looks like a garbage-collected runtime, or an in-memory database that vacuums, or LISP collections of object-trees, etc. There is no point to having a separation between memory and disk anymore, that is the point - "resource allocation is disk persistence" so to speak. It would have required a big-bang rewrite/second-system that is just not possible with the dominance of the existing RAM/storage dichotomy. Or at least a couple killer apps from large vendors/etc that really outperformed what was possible with RAM.

dvas 3y ago
Some support has been added to the Linux kernel like DAX [0] to avoid extra copy operations [0]. SNIA has some interesting talks regarding using non volatile memory and its applications (DBs). Where using pmem and DAX to directly to store logging operations [1]. [0] What is Direct-Access (DAX)? https://kb.pmem.io/faq/100000008-What-is-DAX/ https://kb.pmem.io/faq/100000008-What-is-DAX/ [1] SDC2020: How can Persistent Memory Make Databases Faster, and How Could we go Ahead? https://www.youtube.com/watch?v=rTgITrVhpQM https://www.youtube.com/watch?v=rTgITrVhpQM
- deleted 3y ago
  [deleted]
gavinray 3y ago
Pavlo has some good papers on architecting DB's around PMEM "How to Build a Non-Volatile Memory Database Management System" https://db.cs.cmu.edu/papers/2017/p1753-arulraj.pdf https://db.cs.cmu.edu/papers/2017/p1753-arulraj.pdf
- dvas 3y ago
  Under 3.1 Allocator Interface, there is a great point about reducing wear levelling by rebalancing the B+ tree operations to reduce the number of writes thus extending the lifespan of the hardware. Not much of a concern on the small scale, however in large deployments even single-digit improvements will make a difference. Sometimes it becomes useful to appreciate the hardware and what it is doing, reminds me of the quote. * You don't have to be an engineer to be a racing driver, but you do have to have Mechanical Sympathy *. Jackie Stewart