4 ms·
And the speed of light ensures that memory latencies won't get much better until CPUs are small cubes made of SRAM.
by 0x000xca0xfe 4y ago
And the speed of light ensures that memory latencies won't get much better until CPUs are small cubes made of SRAM.
- actionfromafar 4y agoEhrm. Small spheres of SRAM, if I may.
- speed_spread 4y agoThat's one way of dealing with corner cases.
- amluto 4y agoCome again? Modern servers seem to have about 100ns latency to main memory. The speed of light (actually electrical signals) delay is maybe 1-2ns.
- 0x000xca0xfe 4y agoThanks for the response, this made me dig around some more in this fascinating topic. I found multiple papers that also simply use 100ns for RAM latency (e.g. this one[1]) but in reality it is a lot more complicated than one number. E.g. the time to first word for DDR3 is just above 6ns[2]. I'm not a CPU engineer but it looks like CPUs don't necessarily wait for the entire cache line and can utilize the first word straight away. Also, the signal needs to travel both ways and the traces are not straight lines. I'm not an electronics engineer either so the best number for the maximum DRAM trace lengths I could find are 12-30cm, which would be 1-2ns indeed. This means that there is some room for improvement (in theory). But even with ideal RAM it is physically impossible to beat register access at 0.17ns (i9-13900KS @ 6GHz) or even L1D cache at 0.7ns[3]. What is causing the extra 4-5ns latency of the RAM? Would it be physically possible to remove it? [1] https://www.usenix.org/system/files/conference/hotcloud18/hotcloud18-paper-thomas.pdf https://www.usenix.org/system/files/conference/hotcloud18/ho... [2] https://en.wikipedia.org/wiki/CAS_latency https://en.wikipedia.org/wiki/CAS_latency [3] https://chipsandcheese.com/2022/11/08/amds-zen-4-part-2-memory-subsystem-and-conclusion/ https://chipsandcheese.com/2022/11/08/amds-zen-4-part-2-memo...
- pclmulqdq 4y agoThe extra delay comes from resistance and capacitance in the circuit, which makes electrical signals slow down (this is the ELI5 version). DRAM cells and long wires in the DRAM array are big capacitors and they are driven by circuits with fairly weak output (the resistance). Device physics makes these resistances and capacitances extremely hard to reduce. That is why on-device latency for DRAM has held fairly steady at ~5 ns for a very long time.
- ummonk 4y agoMotherboard manufacturers certainly have to consider DRAM trace lengths when supporting modern DDR5 clock speeds, which are now 6,000 Mhz for top of the line hardware. The CAS latency for RAM is considerably higher and has been decoupled from this - as clock speeds go up CL multipliers go up and the CAS latency ends up being roughly the same - ~10 ns for the best RAM. I'm not sure why CAS latency can't be driven lower. Row selection (tRCD) could be limited by the physics of charging up sense amplifiers, but I don't see why the logic for column address shouldn't be able to run faster. It's probably just something that hasn't been an area of optimization for the simple reason that there are many other much larger sources of latency. And on that note, a CPU won't just jump to accessing some random word from RAM and copying it into a register. It will first look through L1, then L2, and then L3 before seeking the data from RAM. It will then copy an entire cache line into all three caches and load the target word into a register. This entire process results in ~100 ns latency.
- ummonk 4y agoImagine having machine learning model weights just be encoded in blocks of SRAM right on the CPU die.