5 ms·

Dennard scaling for SRAM has certainly halted, as demonstrated by TSMC’s 3nm process vs 5 nm. What’s the likely ETA for DRAM?

wmf 2y ago
Now? Prices have been flat for 15 years and DRAM has been stuck on 10 nm for a while.
- philipkglass 2y ago
  That's overstating the flatness of prices. In 2009, the best price recorded here was 10 dollars per gigabyte: https://jcmit.net/memoryprice.htm https://jcmit.net/memoryprice.htm Recently DDR4 RAM is available at well under $2/GB, some closer to $1/GB.
  - jychang 2y ago
    $1/GB? That's around the price SSDs took over from HDDs...
ksec 2y ago
Not soon as DRAM is mostly on older node. But overall cost reduction of DRAM is moving very very slowly.
Salgat 2y ago
5nm can hold roughly a gigabyte of SRAM on a cpu-sized die, that's around $130/GB I believe. At some point 5nm will be cheap enough that we can start considering replacing DRAM with SRAM directly on the chip (aka L4 cache). I wonder how big of a latency and bandwidth bonus that'd be. You could even go for a larger node size without losing much capacity for half the price.
- bgnn 2y ago
  5nm will never be that cheap. The performance benefit would be easily 2x or more though.
  - Salgat 2y ago
    Even 15 years from now?
- crest 2y ago
  SRAM also requires more power than DRAM and the simple regular structure of SRAM arrays compared to (other) logic makes it possible to get good yield rates through redundancy and error correction codes so you could have giant monolithic dies, but information can't exceed the speed of light in a medium. There just isn't enough time for the signals to propagate to get the latency you expect of a L3 cache out of gigabytes (in relative terms) far away big dies containing gigabytes of SRAM. Also moving that the data would to perform computations without caching would be terrible wasteful given how much energy is needed just to move the data. Instead you would probably end up with something closer to the computing memory concept to map computation to ALUs close to the data with an at least two tier network (on-die, inter-die) to support reductions.
  - Salgat 2y ago
    Oh yeah this would definitely be something like L4 cache rather than L3 like AMD's X3D cpus. The expectation is as an alternative to DRAM (or as a supplement), kind of like what Xeon Phi did.
- deleted 2y ago
  [deleted]
hajile 2y ago
Years ago. DRAM uses a capacitor. Those capacitors essentially hit a hard limit at around 400MHz for our traditional materials a very long time ago. This means that if you need to sequentially read random locations from RAM, you can't do it faster than 400MHz. Our only answer here is better AI prefetchers and less-random memory patterns in our software (the penalty for not prefetching is so great that theoretically less efficient algorithms can suddenly become more efficient if they are simply more predictable). As to capacitor sizes, we've been at the volume limit for quite a while. When the capacitor is discharged, we must amplify the charge. That gets harder as the charge gets weaker and there's a fundamental limit to how small you can go. Right now, each capacitor has somewhere in the range of a mere 40,000 electrons holding the charge. Going lower dramatically increases the complexity of trying to tell the signal from the noise and dealing with ever-increasing quantum effects. Getting more capacitors closer means a smaller diameter, but keeping the same volume means making the cylinder longer. You quickly reach a point where even dramatic increases in height (something very complicated to do in silicon) give only minuscule decreases in diameter.
- aidenn0 2y ago
  If that's the case, why haven't we switched to SRAM? Isn't it only about 4x the price at any given process node?
  - RF_Savage 2y ago
    That 4x the price does also explain why it has not happened.
    - guerrilla 2y ago
      Yeah but why don't we have like 500MB of SRAM per 8GB of RAM. Certain addresses could be faster, in the same way /dev/shm/ is faster.
      - turtles3 2y ago
        This is sort of the role that L3 cache plays already. Your proposal would be effectively an upgradable L4 cache. No idea if the economics on that are worth it vs bigger DRAM so you have less pressure on the nvme disk.
cubefox 2y ago
> Dennard scaling for SRAM has certainly halted, as demonstrated by TSMC’s 3nm process vs 5 nm. I don't think the latter (SRAM capacity remaining the same per area?) has anything to do with Dennard scaling.

wmf 2y ago

Now? Prices have been flat for 15 years and DRAM has been stuck on 10 nm for a while.

philipkglass 2y ago
That's overstating the flatness of prices. In 2009, the best price recorded here was 10 dollars per gigabyte: https://jcmit.net/memoryprice.htm https://jcmit.net/memoryprice.htm Recently DDR4 RAM is available at well under $2/GB, some closer to $1/GB.
- jychang 2y ago
  $1/GB? That's around the price SSDs took over from HDDs...

ksec 2y ago

Not soon as DRAM is mostly on older node. But overall cost reduction of DRAM is moving very very slowly.

Salgat 2y ago

5nm can hold roughly a gigabyte of SRAM on a cpu-sized die, that's around $130/GB I believe. At some point 5nm will be cheap enough that we can start considering replacing DRAM with SRAM directly on the chip (aka L4 cache). I wonder how big of a latency and bandwidth bonus that'd be. You could even go for a larger node size without losing much capacity for half the price.

bgnn 2y ago
5nm will never be that cheap. The performance benefit would be easily 2x or more though.
- Salgat 2y ago
  Even 15 years from now?
crest 2y ago
SRAM also requires more power than DRAM and the simple regular structure of SRAM arrays compared to (other) logic makes it possible to get good yield rates through redundancy and error correction codes so you could have giant monolithic dies, but information can't exceed the speed of light in a medium. There just isn't enough time for the signals to propagate to get the latency you expect of a L3 cache out of gigabytes (in relative terms) far away big dies containing gigabytes of SRAM. Also moving that the data would to perform computations without caching would be terrible wasteful given how much energy is needed just to move the data. Instead you would probably end up with something closer to the computing memory concept to map computation to ALUs close to the data with an at least two tier network (on-die, inter-die) to support reductions.
- Salgat 2y ago
  Oh yeah this would definitely be something like L4 cache rather than L3 like AMD's X3D cpus. The expectation is as an alternative to DRAM (or as a supplement), kind of like what Xeon Phi did.
deleted 2y ago
[deleted]

hajile 2y ago

Years ago. DRAM uses a capacitor. Those capacitors essentially hit a hard limit at around 400MHz for our traditional materials a very long time ago. This means that if you need to sequentially read random locations from RAM, you can't do it faster than 400MHz. Our only answer here is better AI prefetchers and less-random memory patterns in our software (the penalty for not prefetching is so great that theoretically less efficient algorithms can suddenly become more efficient if they are simply more predictable). As to capacitor sizes, we've been at the volume limit for quite a while. When the capacitor is discharged, we must amplify the charge. That gets harder as the charge gets weaker and there's a fundamental limit to how small you can go. Right now, each capacitor has somewhere in the range of a mere 40,000 electrons holding the charge. Going lower dramatically increases the complexity of trying to tell the signal from the noise and dealing with ever-increasing quantum effects. Getting more capacitors closer means a smaller diameter, but keeping the same volume means making the cylinder longer. You quickly reach a point where even dramatic increases in height (something very complicated to do in silicon) give only minuscule decreases in diameter.

aidenn0 2y ago
If that's the case, why haven't we switched to SRAM? Isn't it only about 4x the price at any given process node?
- RF_Savage 2y ago
  That 4x the price does also explain why it has not happened.
  - guerrilla 2y ago
    Yeah but why don't we have like 500MB of SRAM per 8GB of RAM. Certain addresses could be faster, in the same way /dev/shm/ is faster.
    - turtles3 2y ago
      This is sort of the role that L3 cache plays already. Your proposal would be effectively an upgradable L4 cache. No idea if the economics on that are worth it vs bigger DRAM so you have less pressure on the nvme disk.

cubefox 2y ago

> Dennard scaling for SRAM has certainly halted, as demonstrated by TSMC’s 3nm process vs 5 nm. I don't think the latter (SRAM capacity remaining the same per area?) has anything to do with Dennard scaling.