10 ms·

So, it's been a while since I tinkered with hardware, but I'm making the blind assumption that although the bus speed is twice as fast as DDR4, there is also t

So, it's been a while since I tinkered with hardware, but I'm making the blind assumption that although the bus speed is twice as fast as DDR4, there is also twice the read latency, as has always been the case with DDR memory, meaning the net read latency improvement is basically nil. Of course we expect every generation of Double Data Rate memory to double the data rate. But it's always been a bit misleading to say "twice as fast".

mamon 10y ago
Yes, I always considered that inflated bus speeds a scam. It is like having a subway line driving trains every 2 minutes, but only 1 train in 15 taking passengers, rest going completely empty. If you look at CAS latencies [1] then it's clear that memory speeds stagnated a long time ago: [1] https://en.wikipedia.org/wiki/CAS_latency https://en.wikipedia.org/wiki/CAS_latency
- dom0 10y ago
  That's not a very accurate analogy, since sequential reads do improve. CAS latency is one limiting factor on random reads, which has been constant / only slightly improving for a very long time; significant improvements last happened here around with DDR and SDR (i.e. 20 years ago). This is a central reason why the cache size per thread has not changed much in the last 20 years (beside of course fabbing/yield limits). (And because caches are subject to a similar effect as Dennard scaling) (btw. I wonder if it'd have been smarter by the memory industry if they specified latencies in absolute numbers (or relative to the memory array clock, not the bus clock), not bus cycles, since the rising latency numbers have been a marketing issue in every generation.)
  - petra 10y ago
    If cas latency haven't changed for 20 years, and the memory wall in general being a huge pain, does anybody know why didn't some sort of 2.5D memory(even using low-resolution lithography and carriers cheaper than silicon), haven't entered the market ?
    - Retric 10y ago
      Heat, RAM puts out a lot more than you might think. That said, for low power devices like cellphones you can use different packaging and cut latency. Also, AMD for example uses stacked ram (HBM), but that needs very special packaging and active cooling. http://www.amd.com/en-us/innovations/software-technologies/hbm http://www.amd.com/en-us/innovations/software-technologies/h... https://en.wikipedia.org/wiki/High_Bandwidth_Memory https://en.wikipedia.org/wiki/High_Bandwidth_Memory
      - petra 10y ago
        If heat is the problem, and since the memory wall is a 20-year problem, was there some new cooling tech ? or is it just standard industry inertia(and just being satisfied with half solutions) ?
        dom0 10y ago
        No, it's a fundamental issue with the technology. Roughly, the ratio of power density to capacity-that-can-hold-information-for-a-refresh-interval stays more or less constant, independent of fab node. In fact, newer generations (DDR3, DDR4) have been pushing it here (cf. rowhammer). I doubt we will see a drastic improvement in DRAM latency unless the underlying process changes a lot (i.e. different physical storage mechanism or very different manufacturing).
    - theandrewbailey 10y ago
      I think that's what HBM tries to do, with stacking memory dies on top of each other, along with locating them next to its associated processor.
Retric 10y ago
Latency is limited by the speed of electricity which is a large fraction of the speed of light, so unless you move DDR closer to the CPU it physically can't get 2x as fast... ever. light speed * 0.951 (electricity) / 2 (round trip) / 2 feet (CPU to furthest RAM chip along a wire) ~= 250 million cycles per second. However, cache is now huge. So, it's latency is usually less of an issue than sequential reads/writes and total size.
- jjoonathan 10y ago
  I keep hearing this, but it doesn't jibe with the RAM timings I see. tRAS is usually 50ns or so, and propagation delay is 2ns or so (see below). What gives? My suspicion is that amplifying the differential voltages produced by femtocoulombs of charge at 100% reliability is a harder problem than moving the DRAM chips closer to the CPU, and that speed-of-light has gotten the blame in "pop architecture" due to sloppy overgeneralization. 2*2 feet is long for a memory bus, but it's partially cancelled out by the high velocity factor (sorry, "speed of electricity") of 0.951 in parent's calculations. Instead, I'm going with 0.5*2 feet at a velocity factor of 0.5 for a 2ns roundtrip propagation delay.
  - Retric 10y ago
    https://en.wikipedia.org/wiki/CAS_latency https://en.wikipedia.org/wiki/CAS_latency First word latency: DDR3 SDRAM 6.37ns = 1/156,985,871 of a second.
    - jjoonathan 10y ago
      tCL is (very) roughly the RAM equivalent of sequential access time. I'm talking random access time, tRAS, which includes closing the last row (precharging the diff amps) and opening the next (waiting for the amps to stabilize). It's called Random Access Memory, so I think it's more fair to judge it by its random access time.
      - Retric 10y ago
        tCL is very much a round trip latency not just sequential access. If you want sequential access 4 times you can pay for tCL once. https://www.altera.com/ja_JP/pdfs/literature/hb/external-memory/emi_optimizing_efficiency.pdf https://www.altera.com/ja_JP/pdfs/literature/hb/external-mem... DDR is not really setup for random access, because it's setup for talking to L2/L3 cache not registers. tRAS ~= tCL + tRCD + tRP which kind of but not exactly exactly what you want for random access.
adrianratnapala 10y ago
All this means is that we should stop thinking of this stuff as RAM. Only the L1 cache is really RAM. Everything else is just a kind of fast, volatile, solid state disk that just happens to share an address space with the RAM.

mamon 10y ago

Yes, I always considered that inflated bus speeds a scam. It is like having a subway line driving trains every 2 minutes, but only 1 train in 15 taking passengers, rest going completely empty. If you look at CAS latencies [1] then it's clear that memory speeds stagnated a long time ago: [1] https://en.wikipedia.org/wiki/CAS_latency https://en.wikipedia.org/wiki/CAS_latency

dom0 10y ago
That's not a very accurate analogy, since sequential reads do improve. CAS latency is one limiting factor on random reads, which has been constant / only slightly improving for a very long time; significant improvements last happened here around with DDR and SDR (i.e. 20 years ago). This is a central reason why the cache size per thread has not changed much in the last 20 years (beside of course fabbing/yield limits). (And because caches are subject to a similar effect as Dennard scaling) (btw. I wonder if it'd have been smarter by the memory industry if they specified latencies in absolute numbers (or relative to the memory array clock, not the bus clock), not bus cycles, since the rising latency numbers have been a marketing issue in every generation.)
- petra 10y ago
  If cas latency haven't changed for 20 years, and the memory wall in general being a huge pain, does anybody know why didn't some sort of 2.5D memory(even using low-resolution lithography and carriers cheaper than silicon), haven't entered the market ?
  - Retric 10y ago
    Heat, RAM puts out a lot more than you might think. That said, for low power devices like cellphones you can use different packaging and cut latency. Also, AMD for example uses stacked ram (HBM), but that needs very special packaging and active cooling. http://www.amd.com/en-us/innovations/software-technologies/hbm http://www.amd.com/en-us/innovations/software-technologies/h... https://en.wikipedia.org/wiki/High_Bandwidth_Memory https://en.wikipedia.org/wiki/High_Bandwidth_Memory
    - petra 10y ago
      If heat is the problem, and since the memory wall is a 20-year problem, was there some new cooling tech ? or is it just standard industry inertia(and just being satisfied with half solutions) ?
      - dom0 10y ago
        No, it's a fundamental issue with the technology. Roughly, the ratio of power density to capacity-that-can-hold-information-for-a-refresh-interval stays more or less constant, independent of fab node. In fact, newer generations (DDR3, DDR4) have been pushing it here (cf. rowhammer). I doubt we will see a drastic improvement in DRAM latency unless the underlying process changes a lot (i.e. different physical storage mechanism or very different manufacturing).
  - theandrewbailey 10y ago
    I think that's what HBM tries to do, with stacking memory dies on top of each other, along with locating them next to its associated processor.

Retric 10y ago

Latency is limited by the speed of electricity which is a large fraction of the speed of light, so unless you move DDR closer to the CPU it physically can't get 2x as fast... ever. light speed * 0.951 (electricity) / 2 (round trip) / 2 feet (CPU to furthest RAM chip along a wire) ~= 250 million cycles per second. However, cache is now huge. So, it's latency is usually less of an issue than sequential reads/writes and total size.

jjoonathan 10y ago
I keep hearing this, but it doesn't jibe with the RAM timings I see. tRAS is usually 50ns or so, and propagation delay is 2ns or so (see below). What gives? My suspicion is that amplifying the differential voltages produced by femtocoulombs of charge at 100% reliability is a harder problem than moving the DRAM chips closer to the CPU, and that speed-of-light has gotten the blame in "pop architecture" due to sloppy overgeneralization. 2*2 feet is long for a memory bus, but it's partially cancelled out by the high velocity factor (sorry, "speed of electricity") of 0.951 in parent's calculations. Instead, I'm going with 0.5*2 feet at a velocity factor of 0.5 for a 2ns roundtrip propagation delay.
- Retric 10y ago
  https://en.wikipedia.org/wiki/CAS_latency https://en.wikipedia.org/wiki/CAS_latency First word latency: DDR3 SDRAM 6.37ns = 1/156,985,871 of a second.
  - jjoonathan 10y ago
    tCL is (very) roughly the RAM equivalent of sequential access time. I'm talking random access time, tRAS, which includes closing the last row (precharging the diff amps) and opening the next (waiting for the amps to stabilize). It's called Random Access Memory, so I think it's more fair to judge it by its random access time.
    - Retric 10y ago
      tCL is very much a round trip latency not just sequential access. If you want sequential access 4 times you can pay for tCL once. https://www.altera.com/ja_JP/pdfs/literature/hb/external-memory/emi_optimizing_efficiency.pdf https://www.altera.com/ja_JP/pdfs/literature/hb/external-mem... DDR is not really setup for random access, because it's setup for talking to L2/L3 cache not registers. tRAS ~= tCL + tRCD + tRP which kind of but not exactly exactly what you want for random access.

adrianratnapala 10y ago

All this means is that we should stop thinking of this stuff as RAM. Only the L1 cache is really RAM. Everything else is just a kind of fast, volatile, solid state disk that just happens to share an address space with the RAM.