5 ms·
> A shower-thought that popped into my brain this morning is that with ~100ns latency to main memory, that's the equivalent of "only" 10 million IOPS (per chann
by Tuna-Fish 4y ago
> A shower-thought that popped into my brain this morning is that with ~100ns latency to main memory, that's the equivalent of "only" 10 million IOPS (per channel). I say only, because my laptop has an NVMe SSD that can do nearly 1 million IOPS.
That is not how any of this works. Neither RAM nor SSD can get anywhere near their peak throughput if accesses are serialized. If your laptop has DDR5, it can likely do > 1B iops. 100ns is the time to get a single random access to a closed row, but while you are waiting for that access to happen, you can issue new operations every 3ns to different banks and bank groups on the same channel.
- _a_a_a_ 4y agoI don't know how any of this works, and there seem to be a number of experts in this thread so please allow me some very newbie questions. > [can't] get anywhere near their peak throughput if accesses are serialized But first of all, doesn't DRAM have an optimisation for serial accesses, which is (I hope I get this the right way round) to hold the RAS steady then do CAS/CAS/CAS/CAS instead of RAS/CAS + RAS/CAS + RAS/CAS + RAS/CAS? In addition, if you do access RAM serially then there is some interleaving which is automatically inserted so that sequential accesses at some level are sent off to different banks (but I've never been clear if there interleaved at a 64 bytes, to fit a cache line, or at a 4K page size to suit the TLB, or some other size, and I'd really like to know – anyone please?)
- pclmulqdq 4y agoYeah, DRAM controllers optimize for serial access in the mapping from physical address space to DRAM DIMMs, banks, rows, and columns. There's actually a pretty sophisticated algorithm for mapping "physical" addresses to DRAM addresses.
- Tuna-Fish 4y agoSorry, two entirely different definitions of serial, and mine is less useful. What I mean is that if you are doing an infinite pointer hop (that is, load a value, then use the that value as a pointer to load another value, use that as a a pointer... etc), you only get a tiny fraction of the IOPS you get than if you launch 100 different loads to different addresses that you already know. And the inverse of your load to use latency is basically how many IOPS you get if you do that infinite pointer chase. > In addition, if you do access RAM serially then there is some interleaving which is automatically inserted so that sequential accesses at some level are sent off to different banks (but I've never been clear if there interleaved at a 64 bytes, to fit a cache line, or at a 4K page size to suit the TLB, or some other size, and I'd really like to know – anyone please?) They are interleaved at the size of the DRAM row, which is usually 2^16 bits, or 8192 bytes. The reasoning here is that if you are launching a ton of linear accesses, that is, doing CAS/CAS/CAS/CAS... to the same row, you can get full throughput from it. This is because when you have opened a row, you are not reading from the DRAM anymore, the entire row is in a SRAM buffer, and can read full interface throughput from it. Then you only have to open a new row once the current one runs out, which is always in a different bank so can be done in parallel with the read operations from the current row so long as your memory prefetchers are smart enough to start doing it early enough.
- _a_a_a_ 4y agoThanks that's really good, and the interleaving question's been nagging me a long time. There's a lot of depth here when you start asking. Much appreciated.
- riku_iki 4y agoI am curious how this all maps on software. Is it kernel which will will try to do many parallel RAM reads? What is the state if it, and are there any benchmarks..
- jiggawatts 4y agoIt's done almost entirely in the CPU's memory controller. The operating system only controls the page table entries, which is a "high level" concept a layer of abstraction above this.