7 ms·

Optane is magical and was mismanaged by both Micron and Intel. The article doesn't really express what we have lost. The DIMMs were expensive and didn't have

Optane is magical and was mismanaged by both Micron and Intel. The article doesn't really express what we have lost. The DIMMs were expensive and didn't have broad compatibility. They should have made them in a regular DDR3/4/5 format and half the cost of main memory, or 2x to 4x the density for the same price. The density roadmap had them doubling in capacity with each generation. 1TB per DIMM slot. The Register is right in that it was the newest thing to come along in 50 years. Reliability, performance, flexibility, all could have made huge strides under PMEM/Optane architectures. Intel thought it had a golden goose (it did) and that the top profit margin enterprise workloads would flock to it. Intel has everything it needs to succeed except for internal politics. But that is true of all large corpse these days. FPGA fabric built into the Optane DIMM would have been magic.

chrisweekly 4y ago
"corpse" - vs "corps" - kinda funny if it was intentional; maybe funnier if it wasn't
- sitkack 4y ago
  I saw the typo, chuckled and left it in.
- brnt 4y ago
  I mean, corpses do have everything in them to succeed; their cells just can't agree on a actually doing that anymore.
  - civilitty 4y ago
    Not to be pedantic but most corpses alive today don't have anything resembling cell walls let alone cells.
    - MereInterest 4y ago
      To be pedantic, most corpses aren't alive today.
      - Turneyboy 4y ago
        To further the pedantry, parent didn't make any claims about most corpses. Just most corpses that are alive today.
toast0 4y ago
I'm not sure I really understand why Optane went in the DIMM slots instead of the PCIe slots. I know PCIe storage is usually NVMe, but you could also expose the storage as a large memory mapped region. You wouldn't need explicit support from the cpu/chipset/motherboard, although you probably want resizable BAR support, cause the region would presumably be pretty large, and maybe you want to do something for older systems that can't handle a giant region. From the software side of things, PCIe mapped memory is more or less the same as physical memory; from the hardware side, you'd be taking a different path to get there, so you'd have different bottlenecks, but PCIe5 is here with 2x the bandwidth and there's a limited number of applications that really max out their PCIe lanes.
- lproven 4y ago
  [Article author here] FWIW I tried to explain why that was the important aspect of the tech in the article. I also covered it in some depth in my most recent FOSDEM talk, which is linked in the article. As is the script if you don't have time for the video.
- robin_reala 4y ago
  They did do PCIe Optane (e.g. https://www.intel.com/content/www/us/en/products/sku/129833/intel-optane-ssd-905p-series-1-5tb-12-height-pcie-x4-20nm-3d-xpoint/specifications.html https://www.intel.com/content/www/us/en/products/sku/129833/...)
  - toast0 4y ago
    Sure, but looks like that was NVMe, not just a big bunch of bytes, so whatever magical future was enabled by having a big bunch of non-volatile bytes wasn't enabled by this. (unless there was a way to change it into big bunch of bytes mode?)
- MobiusHorizons 4y ago
  My guess is that to get low latency you have to be interacting with a memory controller that expects low latency (like dram) I don’t know how many cycles it would take as a difference, but my impression is that pcie trades throughput for latency compared to dram
Dunedan 4y ago
> The DIMMs were expensive and didn't have broad compatibility. They should have made them in a regular DDR3/4/5 format and half the cost of main memory, or 2x to 4x the density for the same price. Were the prices intentionally high or was that just because they couldn't compete with the economies of scale of DRAM manufacturers?
- mort96 4y ago
  The prices were low if you view them as RAM, the densities were crazy. The prices were extremely high if you view them as a small cache SSD in front of your spinning rust, which was how they were marketed and sold.
MichaelMoser123 4y ago
Is it possible to have DRAM connected to the bus, but without resetting the DRAM upon reboot? If that is possible, then you would have some form of persistence, provided that power never goes off.
- transfire 4y ago
  Sadly, users usually reboot because some poorly written piece of software (probably written in C/C++ or Java) is out of wack and resetting RAM is the only way to fix it. Also keeping RAM running all the time would eat electricity and generate heat. But I like the general idea. It definitely would be a good option.
  - MichaelMoser123 4y ago
    > users usually reboot because some poorly written piece of software You can have that in Java - in any runtime environment, if your java processes keep consuming too much memory - swapping starts, on linux you will have the OOM killer, in the end the thing will reboot. Also operating systems can have bugs, hardware has bugs, nobody is perfect. > Also keeping RAM running all the time would eat electricity and generate heat. In a server room you wouldn't turn out the lights, ever.
  - imtringued 4y ago
    Honestly people reboot not because something is poorly written, they reboot because it is too difficult to understand how to solve the problem so they go with the short term solution instead
  - fuckstick 4y ago
    > out of wack and resetting RAM is the only way to fix it. This is not how this works. It doesn’t matter what language an application is written in, if a process is terminated, that frees all RAM consumed on any commonly used operating system.
rob74 4y ago
The article does however mention the basic issue which led to Optane's failure (at least if I understand it correctly): that all current OSes have a concept of primary and secondary storage, whereas for getting optimal performance out of Optane, you need to have several levels of memory storage (which apparently have to be managed by the OS, as opposed to CPU caches): a program's code can stay in the Optane memory, while the stack and heap have to be in "real" RAM to reduce wear on the Optane memory when the program is executed? So Intel would have had to develop an OS for it, or worked on extending e.g. the Linux kernel in that direction.
- papruapap 4y ago
  I am a bit ignorant in OS stuff but isnt that managed by the filesystem? I think some people were using them as LOG drives in ZFS fs.
  - rob74 4y ago
    That's described in the article, that they could be used in Linux, but only as a volume. Apparently (if I read the article correctly) that wasn't optimal for exploiting the higher performance.
    - voxadam 4y ago
      As I understand it Linux's VFS and Block layer aren't really designed to exploit the type of throughput and latency that Optane provided. What I'm curious about is what happened to all the PMem (Persistent Memory) and NVRAM (Non-volatile RAM) work that went into recent Linux kernels and libraries.
      - fweimer 4y ago
        I thought that with DAX, it was possible to have a real file system, but access to the actual bits goes straight to the backing storage, not the page cache (as it would with traditional block devices). So the block layer and VFS inefficiencies would not necessarily matter, at least if you can use DAX.
hardware2win 4y ago
>They should have made them in a regular DDR3/4/5 format There was/is CXL
ece 4y ago
You're asking for density or cheaper prices, but neither happened. The endurance improved, the IOPS improved, even latency a bit, but density didn't. Or it would've been cheaper than it is. I want to relate this to other forms of storage that went by the wayside, but I hope someone else will try again with a different roadmap and make it work. FGPAs or SOCs with Optane for instant on/low latency applications would've been amazing.
- awiesenhofer 4y ago
  > FGPAs or SOCs with Optane for instant on/low latency applications would've been amazing. Looks like a company called Everspin is working on exactly that: > "will provide FPGA system designers with extremely fast configuration, instant-on boot capability, and rapid updates of critical application parameters such as weighting tables in AI applications." https://www.theregister.com/2022/08/02/kioxia_everspin_persistent_memory/ https://www.theregister.com/2022/08/02/kioxia_everspin_persi...
mistrial9 4y ago
"large corpse" !!
ksec 4y ago
That is just not true, in any shape or form. It wasn't mismanaged at all. Neither Intel or Micron could make it cost competitive. Optane had its shining moment when DRAM and NAND ( both are commodities like ) skyrocketed 3x the price, at one point making Samsung the most profitable company even surpassing Apple. There were roadmaps for power, reliability, performance and density improvement, literally everything except cost. Micron wasn't even the cheer leader, they were only happy to play along with Intel because Intel made the commitment to buy enough capacity for Micron to sustain the business. And Optane could barely compete when DRAM and NAND price were at its peak, making zero profits on most of its revenue. Imagine when DRAM and NAND are back to normal. The future of memory is either extremely low power on Mobile SoC or Ultra High Bandwidth in the server with 128 Cores. Both of these doesn't fit into Optane Roadmap. The market is so small that despite Intel was giving away its optane product and selling it at zero margin they still couldn't fill the minimum orders commitment to Micron and had to paid its penalty.
- funny_falcon 4y ago
  Optane was still cheaper than DRAM modules of same capacity. And were twice larger for same price. Really, I dream of Optane as a "super-fast swap drive" that "extends" RAM (that is what some manufacturers do with NVMe this days). Then it will be ease to have notebook with 256G "RAM" with actual 16G of DRAM and 256G Optane swap. Most users will not notice any difference compared to actual 256G DRAM since Optane is fast enough to make application switch fast. At least my Google Chrome and Intellij Idea could live in peace in such setup, each eating many gigabytes "RAM" since I don't code and watch sites at the same microsecond.
  - arthur2e5 4y ago
    16GB optanes with an ordinary SSD interface are pretty widespread in China and DIYers do use them for swap/pagefile. The problem is that they take up a whole M.2 or NVMe slot with the corresponding number of PCIe lanes… These drives generally come from budget laptops where the optane is originally used as the system disk. Not a good idea anyways and computer shops usually swap them out. You pay about 28 CNY for 16 GB, which is 2x the average SSD price-per-gig.
stjohnswarts 4y ago
I don't think it's "gone forever" it's just gonna sleep for a while like "dumb terminals" now we have a hybrid with tons of cloud apps (and most things headed that way) and local apps and mixes of the two where it makes more sense. The same will happen with something like optane. I'm surprised they didn't push it more in "larger" embedded systems since it would have benefitted well from that NVRAM goodness, probably more so than PCs and cloud compute.
brundolf 4y ago
> all large corpse Freudian slip? :)

chrisweekly 4y ago

"corpse" - vs "corps" - kinda funny if it was intentional; maybe funnier if it wasn't

sitkack 4y ago
I saw the typo, chuckled and left it in.
brnt 4y ago
I mean, corpses do have everything in them to succeed; their cells just can't agree on a actually doing that anymore.
- civilitty 4y ago
  Not to be pedantic but most corpses alive today don't have anything resembling cell walls let alone cells.
  - MereInterest 4y ago
    To be pedantic, most corpses aren't alive today.
    - Turneyboy 4y ago
      To further the pedantry, parent didn't make any claims about most corpses. Just most corpses that are alive today.

toast0 4y ago

I'm not sure I really understand why Optane went in the DIMM slots instead of the PCIe slots. I know PCIe storage is usually NVMe, but you could also expose the storage as a large memory mapped region. You wouldn't need explicit support from the cpu/chipset/motherboard, although you probably want resizable BAR support, cause the region would presumably be pretty large, and maybe you want to do something for older systems that can't handle a giant region. From the software side of things, PCIe mapped memory is more or less the same as physical memory; from the hardware side, you'd be taking a different path to get there, so you'd have different bottlenecks, but PCIe5 is here with 2x the bandwidth and there's a limited number of applications that really max out their PCIe lanes.

lproven 4y ago
[Article author here] FWIW I tried to explain why that was the important aspect of the tech in the article. I also covered it in some depth in my most recent FOSDEM talk, which is linked in the article. As is the script if you don't have time for the video.
robin_reala 4y ago
They did do PCIe Optane (e.g. https://www.intel.com/content/www/us/en/products/sku/129833/intel-optane-ssd-905p-series-1-5tb-12-height-pcie-x4-20nm-3d-xpoint/specifications.html https://www.intel.com/content/www/us/en/products/sku/129833/...)
- toast0 4y ago
  Sure, but looks like that was NVMe, not just a big bunch of bytes, so whatever magical future was enabled by having a big bunch of non-volatile bytes wasn't enabled by this. (unless there was a way to change it into big bunch of bytes mode?)
MobiusHorizons 4y ago
My guess is that to get low latency you have to be interacting with a memory controller that expects low latency (like dram) I don’t know how many cycles it would take as a difference, but my impression is that pcie trades throughput for latency compared to dram

Dunedan 4y ago

> The DIMMs were expensive and didn't have broad compatibility. They should have made them in a regular DDR3/4/5 format and half the cost of main memory, or 2x to 4x the density for the same price. Were the prices intentionally high or was that just because they couldn't compete with the economies of scale of DRAM manufacturers?

mort96 4y ago
The prices were low if you view them as RAM, the densities were crazy. The prices were extremely high if you view them as a small cache SSD in front of your spinning rust, which was how they were marketed and sold.

MichaelMoser123 4y ago

Is it possible to have DRAM connected to the bus, but without resetting the DRAM upon reboot? If that is possible, then you would have some form of persistence, provided that power never goes off.

transfire 4y ago
Sadly, users usually reboot because some poorly written piece of software (probably written in C/C++ or Java) is out of wack and resetting RAM is the only way to fix it. Also keeping RAM running all the time would eat electricity and generate heat. But I like the general idea. It definitely would be a good option.
- MichaelMoser123 4y ago
  > users usually reboot because some poorly written piece of software You can have that in Java - in any runtime environment, if your java processes keep consuming too much memory - swapping starts, on linux you will have the OOM killer, in the end the thing will reboot. Also operating systems can have bugs, hardware has bugs, nobody is perfect. > Also keeping RAM running all the time would eat electricity and generate heat. In a server room you wouldn't turn out the lights, ever.
- imtringued 4y ago
  Honestly people reboot not because something is poorly written, they reboot because it is too difficult to understand how to solve the problem so they go with the short term solution instead
- fuckstick 4y ago
  > out of wack and resetting RAM is the only way to fix it. This is not how this works. It doesn’t matter what language an application is written in, if a process is terminated, that frees all RAM consumed on any commonly used operating system.

rob74 4y ago

The article does however mention the basic issue which led to Optane's failure (at least if I understand it correctly): that all current OSes have a concept of primary and secondary storage, whereas for getting optimal performance out of Optane, you need to have several levels of memory storage (which apparently have to be managed by the OS, as opposed to CPU caches): a program's code can stay in the Optane memory, while the stack and heap have to be in "real" RAM to reduce wear on the Optane memory when the program is executed? So Intel would have had to develop an OS for it, or worked on extending e.g. the Linux kernel in that direction.

papruapap 4y ago
I am a bit ignorant in OS stuff but isnt that managed by the filesystem? I think some people were using them as LOG drives in ZFS fs.
- rob74 4y ago
  That's described in the article, that they could be used in Linux, but only as a volume. Apparently (if I read the article correctly) that wasn't optimal for exploiting the higher performance.
  - voxadam 4y ago
    As I understand it Linux's VFS and Block layer aren't really designed to exploit the type of throughput and latency that Optane provided. What I'm curious about is what happened to all the PMem (Persistent Memory) and NVRAM (Non-volatile RAM) work that went into recent Linux kernels and libraries.
    - fweimer 4y ago
      I thought that with DAX, it was possible to have a real file system, but access to the actual bits goes straight to the backing storage, not the page cache (as it would with traditional block devices). So the block layer and VFS inefficiencies would not necessarily matter, at least if you can use DAX.

hardware2win 4y ago

>They should have made them in a regular DDR3/4/5 format There was/is CXL

ece 4y ago

You're asking for density or cheaper prices, but neither happened. The endurance improved, the IOPS improved, even latency a bit, but density didn't. Or it would've been cheaper than it is. I want to relate this to other forms of storage that went by the wayside, but I hope someone else will try again with a different roadmap and make it work. FGPAs or SOCs with Optane for instant on/low latency applications would've been amazing.

awiesenhofer 4y ago
> FGPAs or SOCs with Optane for instant on/low latency applications would've been amazing. Looks like a company called Everspin is working on exactly that: > "will provide FPGA system designers with extremely fast configuration, instant-on boot capability, and rapid updates of critical application parameters such as weighting tables in AI applications." https://www.theregister.com/2022/08/02/kioxia_everspin_persistent_memory/ https://www.theregister.com/2022/08/02/kioxia_everspin_persi...

mistrial9 4y ago

"large corpse" !!

ksec 4y ago

That is just not true, in any shape or form. It wasn't mismanaged at all. Neither Intel or Micron could make it cost competitive. Optane had its shining moment when DRAM and NAND ( both are commodities like ) skyrocketed 3x the price, at one point making Samsung the most profitable company even surpassing Apple. There were roadmaps for power, reliability, performance and density improvement, literally everything except cost. Micron wasn't even the cheer leader, they were only happy to play along with Intel because Intel made the commitment to buy enough capacity for Micron to sustain the business. And Optane could barely compete when DRAM and NAND price were at its peak, making zero profits on most of its revenue. Imagine when DRAM and NAND are back to normal. The future of memory is either extremely low power on Mobile SoC or Ultra High Bandwidth in the server with 128 Cores. Both of these doesn't fit into Optane Roadmap. The market is so small that despite Intel was giving away its optane product and selling it at zero margin they still couldn't fill the minimum orders commitment to Micron and had to paid its penalty.

funny_falcon 4y ago
Optane was still cheaper than DRAM modules of same capacity. And were twice larger for same price. Really, I dream of Optane as a "super-fast swap drive" that "extends" RAM (that is what some manufacturers do with NVMe this days). Then it will be ease to have notebook with 256G "RAM" with actual 16G of DRAM and 256G Optane swap. Most users will not notice any difference compared to actual 256G DRAM since Optane is fast enough to make application switch fast. At least my Google Chrome and Intellij Idea could live in peace in such setup, each eating many gigabytes "RAM" since I don't code and watch sites at the same microsecond.
- arthur2e5 4y ago
  16GB optanes with an ordinary SSD interface are pretty widespread in China and DIYers do use them for swap/pagefile. The problem is that they take up a whole M.2 or NVMe slot with the corresponding number of PCIe lanes… These drives generally come from budget laptops where the optane is originally used as the system disk. Not a good idea anyways and computer shops usually swap them out. You pay about 28 CNY for 16 GB, which is 2x the average SSD price-per-gig.

stjohnswarts 4y ago

I don't think it's "gone forever" it's just gonna sleep for a while like "dumb terminals" now we have a hybrid with tons of cloud apps (and most things headed that way) and local apps and mixes of the two where it makes more sense. The same will happen with something like optane. I'm surprised they didn't push it more in "larger" embedded systems since it would have benefitted well from that NVRAM goodness, probably more so than PCs and cloud compute.

brundolf 4y ago

> all large corpse Freudian slip? :)