4 ms·

The slowest clock speed for DDR5 is 2,000Mhz or 0.5ns per cycle. At 2/3c, electricity should be able to travel about 10cm max in one cycle. I suppose 4Ghz RAM w

The slowest clock speed for DDR5 is 2,000Mhz or 0.5ns per cycle. At 2/3c, electricity should be able to travel about 10cm max in one cycle. I suppose 4Ghz RAM would be only 5cm. Is having an external unit with much greater than 10cm of copper travel ultimately a limiting factor here with modern day RAM speeds?

ooterness 3y ago
You're conflating latency vs. bandwidth. It is entirely possible to have many bits in transit on a single channel, wired or wireless.
dragontamer 3y ago
Note that modern RAM protocols assume that multiple bits are in the transmission line that is the wires connecting your CPU to RAM. Electrical Engineers since at least DDR2, probably earlier, need to ensure that all lines are delay matched to about 100picoseconds. That is, in DDR2, if DataBit#1 takes 1.1nanoseconds from start-of-wire to end-of-wire, then all other bits must be somewhere between 1.0ns to 1.2ns in length. This requires impedance controlled pcbs from the manufacturer, and length tracking software for PCB design. (Note: advanced PCB CAD software will even recalculate the speed of light across different lengths, as "inner" tracks have more dielectric surrounding them, slowing down electricity's speed, while "outer" tracks are a bit faster) ------- The dielectric of FR4 (the glass/resin used to make PCBs) is what determines the speed of light of the copper actually. And it can be 3.6 or 4.2 or whatever, but the PCB manufacturer will tell you in the PCB specsheets. Copper guides the wave, but the actual wave travels in the dielectric / insulation between the wires (FR4 in modern PCBs, but if you had open-air wire the dielectric would be the open-air surrounding the wires that form the ground-loop return path). There's been a huge amount of improvements to the understanding of electricity in the past 3 decades, and the old circuit models (sometimes still taught today) are kind of obsolete btw. ------- I think I heard that DDR4 is now 10picosecomds (10x more accuracy in delay matching), which is doable with modern CAD and PCB software. I dunno the delay requirements of DDR5. ------------------ https://en.wikipedia.org/wiki/Transmission_line https://en.wikipedia.org/wiki/Transmission_line Anyway, look through this Wikipedia on Transmission Line theory, which is a closer model to how electricity "actually" works (but still isn't perfect). But its the level you need to think of electricity to understand modern CPU-to-RAM connections. In particular, DDR2 / DDR3 / DDR4 / DDR5 connections are almost certainly either Microstrip or Stripline connections, with a fair amount of PCB / Electrical Engineering going into the design to make sure everything works as expected.
- _trampeltier 3y ago
  Long long time ago, when Ethernet used coax cables, the max. length of the segment was not because of the signal strength. It was because with the max. length of an Ethernet segment the first half of the ethernet data packet was on the cable, if now the computer on the other side started to send a packet too, a collision would not be detected, and the packet would be lost.
cmrdporcupine 3y ago
The article seems to reference having external AI etc accelerators colocated with the RAM they use, but then tied to a host machine via CXL. So that makes a certain amount of sense. Latency sensitive stuff happens co-located with the RAM, and then you just take advantage of throughput back to the host. I'm personally now imagining a specialized database appliance which takes the role of the whole of the pager and buffer pool management from a DB (or KV store or whatever); a physical box which ties secondary storage arrays + large quantities of RAM + buffer pool mgmt firmware together on a box, then connect to host system via CXL. Host system does query planning end execution and everything else... Is anybody doing this? Does anybody want to found a startup with me to do this? <sips more and more coffee...>
- pbalcer 3y ago
  There are enterprise database offerings that do something like that with RDMA. I'm almost certain CXL is being looked at for improvements.
- nikriek 3y ago
  I just finished my master’s thesis on database buffer management with CXL [1]. It’s not published due to confidential hardware information, but we are working on a paper. [1] https://hpi.de/rabl/teaching/master-theses/ongoing-masters-theses.html https://hpi.de/rabl/teaching/master-theses/ongoing-masters-t...
  - cmrdporcupine 3y ago
    Very cool. I look forward to the paper! For some reason the whole issue of buffer mgmt is something I nerd out on a bit.
wtallis 3y ago
DRAM latency is tens of clock cycles. There's a minimum of about 10ns of latency happening within the DRAM chip itself, so a slightly longer bus doesn't affect total latency by much.
- xhrpost 3y ago
  Wikipedia is showing a cycle time for CAS of less than 1ns for modern RAM https://en.wikipedia.org/wiki/CAS_latency#Memory_timing_examples https://en.wikipedia.org/wiki/CAS_latency#Memory_timing_exam...
  - wtallis 3y ago
    The figures under 1ns are not latency numbers but merely the duration of a clock cycle.
brucethemoose2 3y ago
This is not exactly the reason, but trace length/loss is a major problem. Laptop SODIMMs are already severely limiting DDR5 bandwidth, server/desktop DIMMs are getting close. Users aren't gonna like it, but unswappable packaged RAM is coming to most CPUs.
- anonymousDan 3y ago
  What do you mean by packaged RAM and what protocol does it use that would be faster than DDR5?
  - brucethemoose2 3y ago
    > What do you mean by packaged RAM Depends. Apple uses LPDDR5X on a cheap substrate, which is basically just RAM chips soldered to a tiny motherboard, and that is already very fast. But it could also mean HBM on an expensive interposer or cheaper Intel EMIB, or something like Samsung's proposed Wide I/O, or even something new like stacked RAM with TSVs. CPU/RAM packaging is getting increasingly complicated. > what protocol does it use that would be faster than DDR5 Depends, but it can just be straight up DDR5. SODIMMS are terrible because they need 1.35V (vs 1.1V stock DDR5) for really slow speeds and terrible ram timings, while soldered DDR5 and LPDDR5X do not. The closer the DDR5 gets to the CPU, the faster, more power efficient and lower latency it gets.

ooterness 3y ago

You're conflating latency vs. bandwidth. It is entirely possible to have many bits in transit on a single channel, wired or wireless.

dragontamer 3y ago

Note that modern RAM protocols assume that multiple bits are in the transmission line that is the wires connecting your CPU to RAM. Electrical Engineers since at least DDR2, probably earlier, need to ensure that all lines are delay matched to about 100picoseconds. That is, in DDR2, if DataBit#1 takes 1.1nanoseconds from start-of-wire to end-of-wire, then all other bits must be somewhere between 1.0ns to 1.2ns in length. This requires impedance controlled pcbs from the manufacturer, and length tracking software for PCB design. (Note: advanced PCB CAD software will even recalculate the speed of light across different lengths, as "inner" tracks have more dielectric surrounding them, slowing down electricity's speed, while "outer" tracks are a bit faster) ------- The dielectric of FR4 (the glass/resin used to make PCBs) is what determines the speed of light of the copper actually. And it can be 3.6 or 4.2 or whatever, but the PCB manufacturer will tell you in the PCB specsheets. Copper guides the wave, but the actual wave travels in the dielectric / insulation between the wires (FR4 in modern PCBs, but if you had open-air wire the dielectric would be the open-air surrounding the wires that form the ground-loop return path). There's been a huge amount of improvements to the understanding of electricity in the past 3 decades, and the old circuit models (sometimes still taught today) are kind of obsolete btw. ------- I think I heard that DDR4 is now 10picosecomds (10x more accuracy in delay matching), which is doable with modern CAD and PCB software. I dunno the delay requirements of DDR5. ------------------ https://en.wikipedia.org/wiki/Transmission_line https://en.wikipedia.org/wiki/Transmission_line Anyway, look through this Wikipedia on Transmission Line theory, which is a closer model to how electricity "actually" works (but still isn't perfect). But its the level you need to think of electricity to understand modern CPU-to-RAM connections. In particular, DDR2 / DDR3 / DDR4 / DDR5 connections are almost certainly either Microstrip or Stripline connections, with a fair amount of PCB / Electrical Engineering going into the design to make sure everything works as expected.

_trampeltier 3y ago
Long long time ago, when Ethernet used coax cables, the max. length of the segment was not because of the signal strength. It was because with the max. length of an Ethernet segment the first half of the ethernet data packet was on the cable, if now the computer on the other side started to send a packet too, a collision would not be detected, and the packet would be lost.

cmrdporcupine 3y ago

The article seems to reference having external AI etc accelerators colocated with the RAM they use, but then tied to a host machine via CXL. So that makes a certain amount of sense. Latency sensitive stuff happens co-located with the RAM, and then you just take advantage of throughput back to the host. I'm personally now imagining a specialized database appliance which takes the role of the whole of the pager and buffer pool management from a DB (or KV store or whatever); a physical box which ties secondary storage arrays + large quantities of RAM + buffer pool mgmt firmware together on a box, then connect to host system via CXL. Host system does query planning end execution and everything else... Is anybody doing this? Does anybody want to found a startup with me to do this? <sips more and more coffee...>

pbalcer 3y ago
There are enterprise database offerings that do something like that with RDMA. I'm almost certain CXL is being looked at for improvements.
nikriek 3y ago
I just finished my master’s thesis on database buffer management with CXL [1]. It’s not published due to confidential hardware information, but we are working on a paper. [1] https://hpi.de/rabl/teaching/master-theses/ongoing-masters-theses.html https://hpi.de/rabl/teaching/master-theses/ongoing-masters-t...
- cmrdporcupine 3y ago
  Very cool. I look forward to the paper! For some reason the whole issue of buffer mgmt is something I nerd out on a bit.

wtallis 3y ago

DRAM latency is tens of clock cycles. There's a minimum of about 10ns of latency happening within the DRAM chip itself, so a slightly longer bus doesn't affect total latency by much.

xhrpost 3y ago
Wikipedia is showing a cycle time for CAS of less than 1ns for modern RAM https://en.wikipedia.org/wiki/CAS_latency#Memory_timing_examples https://en.wikipedia.org/wiki/CAS_latency#Memory_timing_exam...
- wtallis 3y ago
  The figures under 1ns are not latency numbers but merely the duration of a clock cycle.

brucethemoose2 3y ago

This is not exactly the reason, but trace length/loss is a major problem. Laptop SODIMMs are already severely limiting DDR5 bandwidth, server/desktop DIMMs are getting close. Users aren't gonna like it, but unswappable packaged RAM is coming to most CPUs.

anonymousDan 3y ago
What do you mean by packaged RAM and what protocol does it use that would be faster than DDR5?
- brucethemoose2 3y ago
  > What do you mean by packaged RAM Depends. Apple uses LPDDR5X on a cheap substrate, which is basically just RAM chips soldered to a tiny motherboard, and that is already very fast. But it could also mean HBM on an expensive interposer or cheaper Intel EMIB, or something like Samsung's proposed Wide I/O, or even something new like stacked RAM with TSVs. CPU/RAM packaging is getting increasingly complicated. > what protocol does it use that would be faster than DDR5 Depends, but it can just be straight up DDR5. SODIMMS are terrible because they need 1.35V (vs 1.1V stock DDR5) for really slow speeds and terrible ram timings, while soldered DDR5 and LPDDR5X do not. The closer the DDR5 gets to the CPU, the faster, more power efficient and lower latency it gets.