12 ms·
DDR5 Memory Specification Released
- bullen 6y agoHow is just splitting the memory in two separate channels going to make anything faster? How will this affect driver complexity and cache-misses?
- 0-_-0 6y agoSimilarly to how dual channel memory is faster than single channel. Now you can do dual channel with a single stick, or maybe even quad channel with 2 sticks. I believe it should be otherwise transparent to the rest of the system.
- aidenn0 6y agoI think that's wrong. Going from 1 64-bit channel to 2 32-bit channels has the same throughput per clock (or even slightly lower if the per-transaction overhead is paid separately on each channel).
- wmf 6y agoYes, but they also doubled the clock.
- aidenn0 6y agoWhich doesn't answer the question of "why didn't they just double the clock without reducing the bus width" but your answer to the original question sums that up nicely.
- crote 6y agoWell, you could just increase the clock, but it doesn't really gain you anything. G.Skill already has the F4-4800C18D-16GTRS which runs at 4800Mhz. Is it fast? Not really. The F4-3800C14D-16GTZN running at 3800Mhz is actually faster in practice because you're limited by the memory module's latency. Increasing the clock just makes it a lot harder for motherboard and CPU manufacturers to support those speeds, but on its own it doesn't really gain you a lot of speed.
- crote 6y agoBut memory is latency-limited, not bandwidth-limited. The channels are idle quite a lot of the time. So you go from REQUEST1---------RESPONSE1-REQUEST2---------RESPONSE2 to REQUEST1---------RESPONSE1RESPONSE1 REQUEST2---------RESPONSE2RESPONSE2 Each request individually is slightly slower, but the total bandwidth is greatly increased.
- mjw1007 6y agoI think SDRAM can do REQUEST1-REQUEST2---RESPONSE1-RESPONSE2 can't it?
- crote 6y agoYes it can! But, unless I'm mistaken, only within the same bank. This means you can basically only do this for sequential access, not random access. And there's still a non-zero latency between the responses, so it's still slightly slower than doubling the channels.
- aidenn0 6y ago> How is just splitting the memory in two separate channels going to make anything faster? I am not a HW engineer, but: With DDR, the difference of all traces in the same channel (data & clock) has very tight tolerances (on the order of 1/8 or 1/16 of a clock). Having fewer traces per channel may make it easier to route for higher clock speeds. > How will this affect driver complexity and cache-misses? I'm not sure what you mean? The memory controller should abstract almost all of the differences away. There are per-channel configuration settings that are usually configured by the SPD rom, so there will be twice as many to set, but multichannel memory controllers are already a thing, and going from N to 2N of something doesn't really affect software complexity once N is greater than one.
- jmole 6y agoyou can issue two commands simultaneously. yes, the data transfer latency takes twice as long in theory, due to the bus size halving, but with increased clock speeds its not really an issue.
- wmf 6y agoThis is sort of explained in the article. I think they had to use burst length 16 [1] to scale to 6400 MHz, but 16 * 64 bits would be 128 bytes or two cache lines. The whole memory system works in cache lines, so it wouldn't be good if the processor requested one cache line and got two. So they use BL16 with a narrower 32-bit channel to fetch one 64-byte cache line. As long as multiple cores are accessing memory or prefetching is on (it's almost always on), both channels will be utilized so software won't notice. [1] When you do a read operation on DRAM you get a multi-cycle burst of data, not just one word. This amortizes command/address overhead and presumably matches the slow-but-wide internal DRAM array with the fast-but-narrow channel. See https://people.freebsd.org/~lstewart/articles/cpumemory.pdf https://people.freebsd.org/~lstewart/articles/cpumemory.pdf sec. 2.2.
- LargoLasskhyfv 6y agoReminds me a little bit of Virtual Channel Memory (VCM) SDRAM from NEC.
- pavehawk2007 6y agoWonder if this spec will make it easy for embedded systems to catch up. It always seems like they lag behind what's cutting edge. Maybe that's a cost/benefit analysis.
- lnsru 6y agoI have a brand new design with DDR2. I can power memory from existing 1.8V rail, no need for more voltage regulators. And 400 MHz is totally ok for me since I can have whole memory bandwidth for myself, no operating system, etc. And my application is very cutting edge for sure in its domain.
- pavehawk2007 6y agoI'm assuming LPDDR? I think the goals between the two are a bit different. I think that embedded gets quite messy since it's particularly targeted. Thanks for letting me know of your experience!
- lnsru 6y agoMT47 family. Very normal 1.8V DDR2.
- luizfelberti 6y agoDoes this finally address Rowhammer? Ctrl-F on the article yields nothing...
- kube-system 6y agoThat's more of a die-level issue rather than a module-level issue, isn't it?
- dfox 6y agoDDRwhatever is primarily an definition of package level interconnect which has possibility of being used as module level interconnect as one of design constraints. And row hammer and similar things are completely irrelevant for such specifications.
- O5vYtytb 6y agoPossibly with on-die ECC.
- DoctorOetker 6y agodoes current hardware and software already allow keeping counts of detected and corrected ECC errors? is it possible for the OS to attribute it to a specific process? if so it seems like OS'es could track and publically tell on executables
- stefan_ 6y agoRowhammer can be eliminated through RAM encryption, e.g. (Transparent) Secure Memory Encryption in Ryzen processors.
- nullc 6y agoHow does encryption help? When you only need to achieve a 1-bit change it doesn't matter much if the exact change isn't predictable.
- gruez 6y ago>All the while, there are several smaller changes [...], such as [...] on-die ECC This means we don't have to worry about ECC support by CPU/motherboard anymore, right?
- wmf 6y agoI don't assume anything. Intel may find a way to deliberately cripple ECC in the memory controller.
- pedrocx486 6y agoAnd the market may find a way to cripple Intel. I'm looking forward to our ARM future. :-)
- bavell 6y agoThe author clarifies on-die ECC in the comment section: > So on-die ECC is a bit of a mixed-blessing. To answer the big question in the gallery, on-die ECC is not a replacement for DIMM-wide ECC. > On-die ECC is to improve the reliability of individual chips. Between the number of bits per chip getting quite high, and newer nodes getting successively harder to develop, the odds of a single-bit error is getting uncomfortably high. So on-die ECC is meant to counter that, by transparently dealing with single-bit errors. > It's similar in concept to error correction on SSDs (NAND): the error rate is high enough that a modern TLC SSD without error correction would be unusable without it. Otherwise if your chips had to be perfect, these ultra-fine processes would never yield well enough to be usable. > Consequently, DIMM-wide ECC will still be a thing. Which is why in the JEDEC diagram it shows an LRDIMM with 20 memory packages. That's 10 chips (2 ranks) per channel, with 5 chips per rank. The 5th chip is to provide ECC. Since the channel is narrower, you now need an extra memory chip for every 4 chips rather than every 8 like DDR4.
- ksec 6y ago>Combined with die stacking, which allows for up to 8 dies to be stacked as a single chip, then a 40 element LRDIMM can reach an effective memory capacity of 2TB. Or for the more humble unbuffered DIMM, this would mean we’ll eventually see DIMM capacities reach 128GB for your typical dual rank configuration. So on 8 Channel 16 DIMM per socket you could fit a theoretical 32TB of memory. This is insane amount of memory and great for In-Memory Database. ( How is Intel Optane going to compete? ) This makes me wonder, what makes DRAM so expensive? It is still hovering at a median price or around $3/GB compared to NAND which is less than $0.1/GB.
- baybal2 6y ago> This makes me wonder, what makes DRAM so expensive? Greed does. DRAM makers were antitrust busted at least 7 times on my memory in Taiwan, Korea, and USA.
- azinman2 6y agoGreed to me is a funny word here. If illegal antitrust actions are occurring, well that’s obviously bad for the consumer and market, but you say greed as if DRAM makers owe something to the public? They’re a technology business who exist to make money by definition. It’s not a non-profit.
- jeffbee 6y agoThat hasn't been the history of the business. DRAM makers are state-sponsored strategic enterprises which do not act according to rational economic theories.
- black_puppydog 6y agoExactly. Compare that to privately held corporations, which would never think to illegally keep prices up through collaboration, just because they can reasonably expect to get away with it. /s
- mjw1007 6y ago« The big change here is that the command and address bus is being shrunk and partitioned, with the pins being reallocated to the data bus for the second memory channel. Instead of a single 24-bit CA bus, DDR5 will have two 7-bit CA busses, one for each channel » If there are two 32-bit data busses rather than one 64-bit bus, arithmetic suggests they shouldn't need to find extra pins from somewhere. So maybe the rationale for shrinking the CA busses (to 7 rather than 12) is something different?
- tpxl 6y agohttps://images.anandtech.com/doci/15912/DDR5_12.png https://images.anandtech.com/doci/15912/DDR5_12.png DDR4 appears to have had 40 and 32 bit data buses, while this one has 40/40.
- Dylan16807 6y agoIn other words, data bits stay at 64 but ECC bits go from 8 to 16.
- gswdh 6y agoThis may be a stupid point, but, for personal use of computers in their current form, how much memory do you really need? I’m still a little baffled why chrome requires GBs of memory...? Can we have lean software please?
- jiggawatts 6y agoIn a word: No. CPUs have become so fast that relative to their "internal" speeds, RAM is the new hard disk. Databases are becoming in-memory, and going out to fixed storage, even SSD, is an anathema. New applications are not designed to work on data sets bigger than physical memory. Disk-to-disk streaming algorithms are practically unheard of outside of a few niche scenarios. Like I said, even database vendors are moving to in-memory! I love machines with huge amounts of memory. My laptop has 64 GB, and it's great! I can run entire fleets of servers in a local hypervisor. I can load huge blobs of CSV or JSON data into the shell and not have to worry about the 2-5x overhead of the in-memory representation. It'll fit just fine. I can run every "bloated" app at once and still have 50 GB free for "whatever". I've reindexed a database on my laptop in minutes that would have taken days(!) on a production server because it didn't have enough RAM and was thrashing the storage like crazy. Another way to look at it is the "GB per CPU core". With existing AMD EPYC 2 CPUs having 64 cores and 128 threads, the typical 512 GB memory configuration is "only" 8 GB per core, or 4 GB per thread! With a dual-socket server, halve those numbers again. Similarly, mainstream desktop Ryzen CPUs have up to 16 cores, and that's not even talking about the not-so-mainstream Threadripper line. For 4GB per core, you'd need 64 GB. It's likely that AMD will release 24 or 32 core mainstream CPUs in the near future, maybe as soon as 2 years from now when their 5nm products start shipping. I fully expect server CPUs to hit 96-128 cores per socket around the same time frame, or up to 512 hardware threads in a standard two-socket server. Terabytes of memory is going to become "standard" very soon now.
- imtringued 6y agoWow, someone must be running really lean software if they can run all bloated apps at once and only use 14GB. I switched to 32GB because a single application decided it absolutely needed 8GB RAM.
- tlhunter 6y agoBoth DDR4 and DDR5 have the same 288 pinout. Hopefully nothing bad happens if a stick is plugged into the wrong socket.
- wtallis 6y agoSame pin count, but different keying provides a mechanical barrier against inserting the wrong kind of module.
- manigandham 6y agoThere's a notch in the slot that can be changed.
- pizza234 6y ago> Both DDR4 and DDR5 have the same 288 pinout. The article is actually explicit of this not being the case: DDR5 DIMMs: Still 288 Pins, But Changed Pinouts
- simcop2387 6y agoThey'll change the keying so that that can't happen. Same as with DDR2 and DDR3
- anticensor 6y agoWhy not QDR2?
- ATsch 6y agoDespite sounding related, QDR and DDR are mostly unrelated technologies. They are also both poorly named. The real purpose of DDR is not actually to double the data rate, but to halve your clock speed and allow you to use the same frequency for your clock as your data. This mostly benefits signal integrity. QDR is better understood as memory with two ports, one for reading and one for writing, which can be used at the same time. This is a lot more expensive and really doesn't have huge benefits for PCs compared to just adding more channels (as DDR5 does).
- Dylan16807 6y agoWell there's "QDR" memory like you describe, and then there's "QDR" like GDDR5X and GDDR6 have, which is a single port doing 4 transfers per clock. That said DDR5 is solidly on two transfers per clock, so I don't understand the suggestion to call it or use "QDR2" (what's QDR1 for contrast?).
- ATsch 6y agoQDR/QDR2 are real interfaces, although for SRAM. Afaict Micron and Cypress make them for specialialty applications.
- hedora 6y agoI wonder if this spec increases or decrease memory access latency. The article doesn’t say, which makes me suspicious. After all, DDR4 has higher latency than DDR3 running at the same clock speed.
- hvidgaard 6y agoThat is generally the trade off made to get better clock speeds. And the absolute latency is better with once the speed gets sufficiently fast.
- fnord77 6y agoso, will we see mini-ITX mobos that support 128Gb RAM?
- wmf 6y agoYes, although maybe not soon.
- TanjBennett 6y agoSo many misconceptions here about DRAM. DRAM is miraculously cheap. The process probably costs about $1.50 to $2 per GB, the rest is indeed profit. That nets them maybe $4,000 per wafer - and that includes all the testing, slicing, packaging etc. An average CPU chip in your laptop is about the same size as maybe 3 DRAM chips which cost around $20. DRAM runs on a separate process which is dominated by the difficulty of building the capacitors. These are roughly the shape of a pencil (long narrow hexagons) where the central structure which holds the capacitor needs to be etched to perfection in a process that can take days. The transistors underneath are, at that scale, about as large as the chad from a paper hole punch. The capacitors are just about as narrow as material science (limit to voltage arcing through the insulation layers) can make them so there is glacially slow progress in shrinking DRAM further. Meanwhile the transistors are at extreme limits of resolution for liquid immersion processing, as also are the lines needed to join the rows and columns. Getting those perfect requires very specialized and competent processing. They are not easy, second rate circuits. They are a completely separate branch of the silicon world. Unfortunately since they don't scale much any more, current design methods were mature 8 years ago, the only way you get more of them is to build new factories. That means it is a seller's market in a game where building another fab costs $10B and will only succeed if staffed by really expert people. So, it is generally profitable. The 3 vendors cannot easily undercut each other since they all have roughly the same limits, and any attempt to flood the market takes 4 years to build and everyone can see it coming. So there you are. DRAM is the pivotal technology of the current computer era. Fixing that will most likely require breakthroughs in fundamental memory technology - or a reason for demand to collapse.
- legulere 6y agoAre there any changes in there for bulk memory operations such as copying or zeroing?