5 ms·

I can already imagine storage systems where the blocks are just written to memory before acknowledging as a synchronous commit, and reads occurring completely f

I can already imagine storage systems where the blocks are just written to memory before acknowledging as a synchronous commit, and reads occurring completely from this cache or for one direct hop from this cache. It's going to be amazing in the future as this tech matures. A persistent RAM will change a lot of our architectures for the better.

wmf 8y ago
That's pretty much how SANs used to work.
- reacharavindh 8y ago
  If you're referring to NetApp's use of NVRAM, to take in writes, and acknowledging the IO before actually writing to disk, it is not that simple. They do NVRAM mirroring to save those writes to another partner's NVRAM for HA. This is a network tax on core IO path. If there is a mature Optane DIMM, it may not be necessary to do the network dance in IO path. I sincerely hope no storage company acknowledges a write after just writing to volatile main memory. That's a recipe for disaster when a node goes down.
  - voodootrucker 8y ago
    I don't know if it's SAN specific, but while waiting for XPoint, several vendors went to battery-backed DRAM. They confirm writes when in RAM, then whenever power is lost all RAM is dumped to flash, then loaded back up on boot: https://en.wikipedia.org/wiki/NVDIMM https://en.wikipedia.org/wiki/NVDIMM Fortunately XPoint should solve this.
  - atomicUpdate 8y ago
    > I sincerely hope no storage company acknowledges a write after just writing to volatile main memory. That's a recipe for disaster when a node goes down. Sorry to break this to you, but this is exactly what every major storage product does. Data will be mirrored to separate DDR behind and then good status is given to the host. The data will be destaged at some point later when cache space is required for some other operation. The data itself is safe as long as the battery backup (or capacitor for smaller systems) is charged enough to handle a power outage. The storage system knows the battery levels and may not allow a write cache if there isn't enough supplemental power to destage the full write cache in the event of a power loss.
    - heavenlyblue 8y ago
      This is just mixing up the issues, same as saying "what if the HDD that acknowledged the write suddenly catastrophically failed"?
- adrianmonk 8y ago
  The idea goes back a long time. There used to be a product called Legato Prestoserve that used battery-backed SRAM. It was available in the early 1990s, maybe late 1980s. Here's some documentation about the version of it that Sun Microsystems licensed and sold: https://docs.oracle.com/cd/E19957-01/801-4896-11/Presto.chp1.html https://docs.oracle.com/cd/E19957-01/801-4896-11/Presto.chp1... It says: "Each NVSIMM contains memory, a battery, and power controller circuitry, which ensure that the memory is not lost when the system is shut-down or halts because of an abnormal condition." "Synchronous write requests to disk are intercepted, and the data is stored in non-volatile memory"
  - jiveturkey 8y ago
    ah ... ye olde NVSIMM!
loeg 8y ago
This is already done in enterprise storage systems, they just use a non-Optane persistent journal technology. NVRAM solutions exist today that are just DRAM with a supercap and/or battery plus a flash SSD. Or PCIe cards. Or ordinary RAM plus specialized firmware that kicks in on brown out and copies RAM out to SSD. Optane looks neat, but it's kind of expensive and has relatively low write endurance compared to ordinary DRAM (from an enterprise filesystem journal perspective).

wmf 8y ago

That's pretty much how SANs used to work.

reacharavindh 8y ago
If you're referring to NetApp's use of NVRAM, to take in writes, and acknowledging the IO before actually writing to disk, it is not that simple. They do NVRAM mirroring to save those writes to another partner's NVRAM for HA. This is a network tax on core IO path. If there is a mature Optane DIMM, it may not be necessary to do the network dance in IO path. I sincerely hope no storage company acknowledges a write after just writing to volatile main memory. That's a recipe for disaster when a node goes down.
- voodootrucker 8y ago
  I don't know if it's SAN specific, but while waiting for XPoint, several vendors went to battery-backed DRAM. They confirm writes when in RAM, then whenever power is lost all RAM is dumped to flash, then loaded back up on boot: https://en.wikipedia.org/wiki/NVDIMM https://en.wikipedia.org/wiki/NVDIMM Fortunately XPoint should solve this.
- atomicUpdate 8y ago
  > I sincerely hope no storage company acknowledges a write after just writing to volatile main memory. That's a recipe for disaster when a node goes down. Sorry to break this to you, but this is exactly what every major storage product does. Data will be mirrored to separate DDR behind and then good status is given to the host. The data will be destaged at some point later when cache space is required for some other operation. The data itself is safe as long as the battery backup (or capacitor for smaller systems) is charged enough to handle a power outage. The storage system knows the battery levels and may not allow a write cache if there isn't enough supplemental power to destage the full write cache in the event of a power loss.
  - heavenlyblue 8y ago
    This is just mixing up the issues, same as saying "what if the HDD that acknowledged the write suddenly catastrophically failed"?
adrianmonk 8y ago
The idea goes back a long time. There used to be a product called Legato Prestoserve that used battery-backed SRAM. It was available in the early 1990s, maybe late 1980s. Here's some documentation about the version of it that Sun Microsystems licensed and sold: https://docs.oracle.com/cd/E19957-01/801-4896-11/Presto.chp1.html https://docs.oracle.com/cd/E19957-01/801-4896-11/Presto.chp1... It says: "Each NVSIMM contains memory, a battery, and power controller circuitry, which ensure that the memory is not lost when the system is shut-down or halts because of an abnormal condition." "Synchronous write requests to disk are intercepted, and the data is stored in non-volatile memory"
- jiveturkey 8y ago
  ah ... ye olde NVSIMM!

loeg 8y ago

This is already done in enterprise storage systems, they just use a non-Optane persistent journal technology. NVRAM solutions exist today that are just DRAM with a supercap and/or battery plus a flash SSD. Or PCIe cards. Or ordinary RAM plus specialized firmware that kicks in on brown out and copies RAM out to SSD. Optane looks neat, but it's kind of expensive and has relatively low write endurance compared to ordinary DRAM (from an enterprise filesystem journal perspective).