5 ms·

Is there a theoretical "practical" limit for how fast these things can physically get (in this particular form factor) My understanding is that NVMe interface

Is there a theoretical "practical" limit for how fast these things can physically get (in this particular form factor) My understanding is that NVMe interface is pretty much as close as you can get to the CPU without being integrated. Is there a world where these things can operate as fast as RAM?

baybal2 5y ago
Flash chips themselves can't operate as fast as RAM. You can get as much bandwidth as RAM, but you can not get it as physically fast as RAM i.e. faster than 200ns
- anarazel 5y ago
  Why? DMA can do transfers to/from CPU caches.
  - baybal2 5y ago
    NAMD flash cells themselves can't charge/discharge faster
- m4rtink 5y ago
  What about those non volatile RAM technologies (IIRC called 3D XPoint) Intel is using for their Optane stuff ? It seems to be kinda in between RAM and flash spec wise.
  - baybal2 5y ago
    Intel-Micron tried, we seen how it went. It wasn't really that fast, or wear resistant to replace flash.
    - LinAGKar 5y ago
      Are you sure? That's not what I've heard. It's just really expensive, and not very dense.
      - baybal2 5y ago
        Rephrasing: the speed, and write endurance were sufficiently jaw dropping enough to beat flash's density, and cost.
  - Der_Einzige 5y ago
    I actually was one of the people who did performance benchmarking of 3D XPoint before it came out. In app direct mode, you can maybe eek out 70% of the throughout and about 3x worse latency. Also, not all apps support app direct mode. Many customers try to use 3D XPoint as another part of the cache hierarchy in-between regular ssd and ram. It's actually pretty neat for faas workloads which want containers to be "warm" rather than "hot" or "cold"...
    - wtallis 5y ago
      To clarify for readers who aren't current on all the lingo: app direct mode refers to one of the modes for using the Optane persistent memory modules that connect to the CPU's DRAM controller. It doesn't apply to the Optane NVMe SSDs that use PCI Express and require the software overhead of a traditional IO stack. In a few years, something like CXL may allow for something close to the app direct mode to be usable on persistent memory devices with an SSD-like form factor, but we're not there yet.
  - the8472 5y ago
    Optane comes in two flavors. As NVMe storage and as NVDIMM. The former sits a bit below flash in terms of latency. The latter sits between a bit above DRAM in terms of latency and is byte-addressable.
digikata 5y ago
NVMe is built on PCIe, so latency wise, they will be limited by the PCIe latency (roughly an order of magnitude slower than the memory bus though pcie 5 may be smaller than that) + media latency. Throughput wise, they are limited by the number of PCIe lanes that the controller supports, and the system it plugs into has been sized to allocate.
- MonaroVXR 5y ago
  NVME isn't build on PCIe. It utilizes it and according to the Linus Tech Tips video of the M1 Mac testing. That this isn't always the case.
dragontamer 5y ago
> My understanding is that NVMe interface is pretty much as close as you can get to the CPU without being integrated. The best for consumer technology, yes. But future I/O protocols continue to improve and get better. NVidia / IBM's collaboration on OpenCAPI (which is deployed in Summit as a CPU/GPU interface) has 300GBps I/O between the CPU and GPU, far faster than NVMe speeds (and even DDR4 RAM bandwidth). And future chips may go even faster. I/O is probably one of the fastest growing aspects of a modern computer. PCIe 5.0, CXL, OpenCAPI, etc. etc. Lots and lots of new technology coming into play here. There are even some products on making Flash-RAM work on the DDR4 interface. Non-volatile memory is what that's called. Intel's Optane works pretty well on that. Its not very mainstream but I hear reports that its impressive (slower than real RAM of course, but storage that has the bandwidth of RAM is still cool). > Is there a world where these things can operate as fast as RAM? Well... yesish. Flash is a kind of RAM (random access memory). To answer your fundamental question though: No. Flash is fundamentally higher latency than DRAM (aka: DDR4). But with enough parallelism / big enough flash arrays (or DRAM arrays), you can continue to get higher and higher bandwidth. At the top-of-the-line is SRAM, the RAM used to make L1 cache and registers inside of CPUs. This is very expensive and rarely used. -------- Then you've got various non-mainstream RAMs: FRAM, MRAM, Optane, etc. etc.
- senbarryobama 5y ago
  How did you get this knowledge?
gameswithgo 5y ago
Just put a battery on ram and you have an sdd as fast as ram, in principle. The big thing about SSDs is that while sequential reads and writes have steadily improved, many workloads have not. From a cheap SSD to the best, there is very minimal difference in things like "how fast does my computer boot", "how fast does my game or game level load" and "how fast does visual studio load", or "how fast does gcc compile"
- kmonsen 5y ago
  Is your second paragraph really true? These exact worlds are the reasons many have upgraded their ssd, the PS5 will only let you start new games from they built in fast hard drive for example.
  - wincy 5y ago
    The PS5s operating system and even how they package game files had to be developed from the ground up to realize these gains. There’s a good technical talk about it from one of Sony’s engineers. https://m.youtube.com/watch?v=ph8LyNIT9sg https://m.youtube.com/watch?v=ph8LyNIT9sg
    - emkoemko 5y ago
      so Sony made their own OS for PS5? they are not using FreeBSD anymore?
      - wtallis 5y ago
        They might still be using BSD stuff, but they developed new IO and compression offload functionality that doesn't match any off the shelf capabilities I'm aware of.
        touisteur 5y ago
        Might be close to spdk + compress accelerators?
        5y ago

baybal2 5y ago

Flash chips themselves can't operate as fast as RAM. You can get as much bandwidth as RAM, but you can not get it as physically fast as RAM i.e. faster than 200ns

anarazel 5y ago
Why? DMA can do transfers to/from CPU caches.
- baybal2 5y ago
  NAMD flash cells themselves can't charge/discharge faster
m4rtink 5y ago
What about those non volatile RAM technologies (IIRC called 3D XPoint) Intel is using for their Optane stuff ? It seems to be kinda in between RAM and flash spec wise.
- baybal2 5y ago
  Intel-Micron tried, we seen how it went. It wasn't really that fast, or wear resistant to replace flash.
  - LinAGKar 5y ago
    Are you sure? That's not what I've heard. It's just really expensive, and not very dense.
    - baybal2 5y ago
      Rephrasing: the speed, and write endurance were sufficiently jaw dropping enough to beat flash's density, and cost.
- Der_Einzige 5y ago
  I actually was one of the people who did performance benchmarking of 3D XPoint before it came out. In app direct mode, you can maybe eek out 70% of the throughout and about 3x worse latency. Also, not all apps support app direct mode. Many customers try to use 3D XPoint as another part of the cache hierarchy in-between regular ssd and ram. It's actually pretty neat for faas workloads which want containers to be "warm" rather than "hot" or "cold"...
  - wtallis 5y ago
    To clarify for readers who aren't current on all the lingo: app direct mode refers to one of the modes for using the Optane persistent memory modules that connect to the CPU's DRAM controller. It doesn't apply to the Optane NVMe SSDs that use PCI Express and require the software overhead of a traditional IO stack. In a few years, something like CXL may allow for something close to the app direct mode to be usable on persistent memory devices with an SSD-like form factor, but we're not there yet.
- the8472 5y ago
  Optane comes in two flavors. As NVMe storage and as NVDIMM. The former sits a bit below flash in terms of latency. The latter sits between a bit above DRAM in terms of latency and is byte-addressable.

digikata 5y ago

NVMe is built on PCIe, so latency wise, they will be limited by the PCIe latency (roughly an order of magnitude slower than the memory bus though pcie 5 may be smaller than that) + media latency. Throughput wise, they are limited by the number of PCIe lanes that the controller supports, and the system it plugs into has been sized to allocate.

MonaroVXR 5y ago
NVME isn't build on PCIe. It utilizes it and according to the Linus Tech Tips video of the M1 Mac testing. That this isn't always the case.

dragontamer 5y ago

> My understanding is that NVMe interface is pretty much as close as you can get to the CPU without being integrated. The best for consumer technology, yes. But future I/O protocols continue to improve and get better. NVidia / IBM's collaboration on OpenCAPI (which is deployed in Summit as a CPU/GPU interface) has 300GBps I/O between the CPU and GPU, far faster than NVMe speeds (and even DDR4 RAM bandwidth). And future chips may go even faster. I/O is probably one of the fastest growing aspects of a modern computer. PCIe 5.0, CXL, OpenCAPI, etc. etc. Lots and lots of new technology coming into play here. There are even some products on making Flash-RAM work on the DDR4 interface. Non-volatile memory is what that's called. Intel's Optane works pretty well on that. Its not very mainstream but I hear reports that its impressive (slower than real RAM of course, but storage that has the bandwidth of RAM is still cool). > Is there a world where these things can operate as fast as RAM? Well... yesish. Flash is a kind of RAM (random access memory). To answer your fundamental question though: No. Flash is fundamentally higher latency than DRAM (aka: DDR4). But with enough parallelism / big enough flash arrays (or DRAM arrays), you can continue to get higher and higher bandwidth. At the top-of-the-line is SRAM, the RAM used to make L1 cache and registers inside of CPUs. This is very expensive and rarely used. -------- Then you've got various non-mainstream RAMs: FRAM, MRAM, Optane, etc. etc.

senbarryobama 5y ago
How did you get this knowledge?

gameswithgo 5y ago

Just put a battery on ram and you have an sdd as fast as ram, in principle. The big thing about SSDs is that while sequential reads and writes have steadily improved, many workloads have not. From a cheap SSD to the best, there is very minimal difference in things like "how fast does my computer boot", "how fast does my game or game level load" and "how fast does visual studio load", or "how fast does gcc compile"

kmonsen 5y ago
Is your second paragraph really true? These exact worlds are the reasons many have upgraded their ssd, the PS5 will only let you start new games from they built in fast hard drive for example.
- wincy 5y ago
  The PS5s operating system and even how they package game files had to be developed from the ground up to realize these gains. There’s a good technical talk about it from one of Sony’s engineers. https://m.youtube.com/watch?v=ph8LyNIT9sg https://m.youtube.com/watch?v=ph8LyNIT9sg
  - emkoemko 5y ago
    so Sony made their own OS for PS5? they are not using FreeBSD anymore?
    - wtallis 5y ago
      They might still be using BSD stuff, but they developed new IO and compression offload functionality that doesn't match any off the shelf capabilities I'm aware of.
      - touisteur 5y ago
        Might be close to spdk + compress accelerators?
      - 5y ago