3 ms·
You would need a new CPU that doesn't yet exist to address 32TB of memory per socket. Existing parts can address 4TB. x86-64 has an ultimate system limit of 25
by jeffbee 6y ago
You would need a new CPU that doesn't yet exist to address 32TB of memory per socket. Existing parts can address 4TB. x86-64 has an ultimate system limit of 256TB, due to its 48-bit virtual address space.
Also worth considering that 32TB of DRAM would draw over 12kW, just sitting there.
- kryptiskt 6y agoIce Lake added 5-level paging and 56-bit pointers, so they can address 128 PB: https://en.wikipedia.org/wiki/Intel_5-level_paging https://en.wikipedia.org/wiki/Intel_5-level_paging
- messe 6y agoHey, I'm a bit late, but it's 57-bits. On 64-bit systems page tables have 512 = 2^9 entries and page frames are 4096 = 2^12 bytes long. So the number of bits in an n-level system is 9n + 12. 9x5 + 12 = 57 On 32-bit systems the number of PT entries was 2^10, which is why they could have a 2-level system. 10x2 + 12 = 32.
- messe 6y ago> You would need a new CPU that doesn't yet exist to address 32TB of memory per socket. Existing parts can address 1TB (Intel) or 4TB (AMD). x86-64 has an ultimate system limit of 256TB, due to its 48-bit virtual address space. That's the virtual address space. A page table entry has enough bits to have a 64-bit physical address space, it just wouldn't be able to have it all mapped at once in the same virtual address space. Although CPUs don't have 64 physical address lines yet, there's nothing fundamental in the x86 architecture preventing them from doing so. Intel have already implemented 5-level paging, which would give a 2^57 bit virtual address. Also, this wouldn't be the first time that the x86 has supported more physical memory than virtual. PAE allowed for 64GB of RAM on a system with a 32-bit virtual address space.
- jeffbee 6y agoIt costs them money to add physical address pins, though. While we're here I hope anyone can explain to me why Cooper Lake Xeon parts are listed as supporting 4.5TB of DDR4. Answering self: the 4.5TB support is mostly Optane memory, not DDR4.
- Kuinox 6y agohttps://yourdatafitsinram.net/ https://yourdatafitsinram.net/ IBM Power System E980, max ram: 64TB.
- jeffbee 6y agoSure but it only supports that with 16 CPU sockets.
- Dylan16807 6y agoYou need a CPU that doesn't exist yet just because it's DDR5! Expanding from 48 toward 64 bits isn't difficult.
- gautamcgoel 6y agoWhoa! 12kW?? Just from RAM? I didn't realize RAM drew so much power.
- wahern 6y agoCrucial says approximately 3W per 8GB: https://www.crucial.com/support/articles-faq-memory/how-much-power-does-memory-use https://www.crucial.com/support/articles-faq-memory/how-much... $ echo '(32 * 2^40) / (8 * 2^30) * 3' | bc 12288
- osamagirl69 6y agoFor what its worth, a significant fraction of that is in the communication interfaces, and not from the ram itself, and there have already been significant process improvements to reduce ram power consumption. A modern 256GB RDIMM draws a heck of a lot less then 50w--I have never measured but based on the thermal solution I would say closer to 5w
- jeffbee 6y agoI don't see how that could be true. On my server right here with a Xeon Silver 4114 it is measuring the power consumption of the memory at ~75W for 256GB.
- osamagirl69 6y agoLike I said, ram power consumption does not scale linearly with capacity due to the significant overhead from the IO. A single 128GB stick will draw much less than 16x16gb sticks (not sure why you are using 256GB on a 4114, it has 6 memory channels so surely you have 288GB?) Here is the datasheet for a 128GB dimm from 2017 [1], which shows 3.4A IDD0 (normal operation) on the 1.2V rail at the highest speed of DDR4-2666, and 0.2A on the 2.5V precharge rail for a total of just over 6W. Also worth noting is that is a LRDIMM, which draws more power from the DC rails due to the additional buffering. A normal RDIMM draws a bit less static power. Compare to a manual for a similar vintage 32GB stick [2], which consumes 2A on the 1.2V rail and 0.1A on the precharge rail for a total of a bit under 3w. One quarter the capacity, but still half of the power draw. [1] https://www.samsung.com/semiconductor/global.semi/file/resource/2018/04/TSV_128GB_only_DDR4_8Gb_B_die_LRDIMM_Rev1.6_Jun.17.pdf https://www.samsung.com/semiconductor/global.semi/file/resou... [2]https://static6.arrow.com/aropdfconversion/d3b3ce1d78b0ad7d358ed18e4cf1b07ece10e894/ds_32gb_ddr4_8gb_bdie_loadreduceddimm_rev10-0.pdf https://static6.arrow.com/aropdfconversion/d3b3ce1d78b0ad7d3...
- mycall 6y ago> You would need a new CPU that doesn't yet exist to address 32TB of memory per socket. I thought RISC-V could with the RV128 ISA.