3 ms·

Yes. Persistent memory: http://pmem.io http://pmem.io. This a storage medium accessible like memory and much faster than traditional disks. It blurs the line be

Yes. Persistent memory: http://pmem.io http://pmem.io. This a storage medium accessible like memory and much faster than traditional disks. It blurs the line between memory and disk and will fundamentally alter the way we build systems. For example you may no longer need a leveldb or sqlite disk store, you could just use plain data structures in persistent memory. An early version is currently available with i3 instances on Amazon. Expect to hear more about this in the next few years.

jawilson2 9y ago
I'm more interested in what is actually being used right now, e.g. if I am deploying to a bunch of Skylake processors, what has changed?
gwbas1c 9y ago
Oh gosh no! We already have something similar with memory mapped files. The original .doc format was just Word's data structures shoved in a memory-mapped file. That's why it was so hard to make 3rd party applications compatible. Serialization, like xml or SQLite, is needed to ensure that a file format is interoperable among different programs and different versions of your program. How can you add a field to a data structure in version 2 when your data structures are so tightly coupled to your persistent data? Furthermore, things like SQLite offer indexing that lets you find data without needing to traverse all your RAM in a foreach loop. It abstracts away your application's data's format from the algorithm needed to access it quickly. And finally, what happens when your program crashes? What if your data structures are in a bad state, or corrupt? What about transactional integrity? How do you abstract your pointers, because the addresses of your structures will change the next time around. I suspect that SQLite (and similar) will have updates that take advantage of persistent memory. (Edit) I also suspect that persistent memory will help lower power consumption. A device could go into a kind of a sleep mode much more easily if it didn't need to page in and out its memory.

jawilson2 9y ago

I'm more interested in what is actually being used right now, e.g. if I am deploying to a bunch of Skylake processors, what has changed?

gwbas1c 9y ago

Oh gosh no! We already have something similar with memory mapped files. The original .doc format was just Word's data structures shoved in a memory-mapped file. That's why it was so hard to make 3rd party applications compatible. Serialization, like xml or SQLite, is needed to ensure that a file format is interoperable among different programs and different versions of your program. How can you add a field to a data structure in version 2 when your data structures are so tightly coupled to your persistent data? Furthermore, things like SQLite offer indexing that lets you find data without needing to traverse all your RAM in a foreach loop. It abstracts away your application's data's format from the algorithm needed to access it quickly. And finally, what happens when your program crashes? What if your data structures are in a bad state, or corrupt? What about transactional integrity? How do you abstract your pointers, because the addresses of your structures will change the next time around. I suspect that SQLite (and similar) will have updates that take advantage of persistent memory. (Edit) I also suspect that persistent memory will help lower power consumption. A device could go into a kind of a sleep mode much more easily if it didn't need to page in and out its memory.