4 ms·

> Is it that the CPU would be unable to effectively use all that bandwidth due to less parallelism compared with a GPU? Yes, but also it's due to simple physic

> Is it that the CPU would be unable to effectively use all that bandwidth due to less parallelism compared with a GPU? Yes, but also it's due to simple physical limitations. If you look at how close CPUs are to RAM as well as the number of PCB lines coming out of CPUs, it becomes pretty readily apparent that there's not much room for additional lines. You have to recognize that each of those lines is creating a magnetic field, the closer you stack them, the more likely cross talk is. We could continue to lower the voltage on the memory lanes so you can pack them closer together, but then you have to start worrying about environmental interference on the lines. It likely won't be too long in the future (if it's not already here) that we'll start seeing shielding for the memory lanes so they can be packed closer together and more can be added. It wouldn't shock me if this became a part of DDR6. It also wouldn't shock me to start seeing more complex memory controllers doing things like lz4 compression to eck out even higher bandwidths (particularly when the memory is highly uniform). We are sort of seeing the precursor of this in DDR5 with commands to set whole blocks to a given value.

CoastalCoder 3y ago
I'm curious if anytime soon we'll start seeing optical interconnections between CPUs and on-device / same-chasis memory. I'm not sure what that would add for latency and cost, but I'm guessing that with optical wave guides, crosstalk isn't a concern. At least for the optical part of the data path.
- kimixa 3y ago
  The speed of light in an optical fiber is still only about 2*10^8 m/s - ddr5 6000's 3ghz clock means it still only travels 6.7cm/clock - so there'll still be difficulties around latency if it's not "close", and that's ignoring any issues of actually getting optical transceivers on the die itself.
  - billti 3y ago
    It blew my mind when I learned that even if you could hand wire conceptually perfect switches and wires, once your board was over a few inches, the laws of physics (e.g., the speed of light) would mean you still couldn’t beat a modern CPU for clock speed. It really is insane the level at which current hardware works.
    - josephg 3y ago
      Yeah. Any time I wait for my computer to do something I think about this. In the 4 seconds discord took to open, my computer could do what, 5 billion operations per second per core? Even single threaded that’s 20 billion steps? And my internet connection could have delivered 600 million bytes of data - enough for a complete copy of windows xp. It’s wild.
      - steve1977 3y ago
        It’s wild and a sign how terribly inefficient most software is today
- trealira 3y ago
  > I'm curious if anytime soon we'll start seeing optical interconnections between CPUs and on-device / same-chasis memory. Isn't that kind of what those APUs sold by AMD are? And SoCs? I could be misunderstanding you, though.
- ta988 3y ago
  They are coming yes. They may not bring much in term of bandwidth per line, but will allow much more lines and entirely different geometries for the cpus allowing chips to be cooled on both sides etc.
AnthonyMouse 3y ago
> You have to recognize that each of those lines is creating a magnetic field, the closer you stack them, the more likely cross talk is. PC desktops have 2 memory channels. EPYC servers have 12 channels per socket. It's not the physics, it's that ~8-core processors generally aren't bounded by memory bandwidth and it's not worth the cost of increasing the number of pins. The server sockets do it because they have 128-core processors. If this changes what we may start seeing is CPUs (or iGPUs) with a few GB of HBM as L4 cache.
- simcop2387 3y ago
  Intel is doing this today in the Xeon MAX line, up to 64gb of hbm ram, you don't even necessarily need ddr5 on them. serve the home has a good article about them but it's all very new so it's hard to say how it'll shake out.
- dathinab 3y ago
  > not the physics no physics matter because it makes it much harder and in turn more costly to the more channels you have to have high quality reliable connections server RAM is for example also slower to compensate this e.g. recent EPYC processors run at up to 4800MT/s but a Ryzen 9 runs at up to 5200MT/s (without overclocking, with running at 6000MT/s isn't that uncommon) server also tend to always use EEC memory so it's a benefit/cost balance calculation
  - AnthonyMouse 3y ago
    > server RAM is for example also slower to compensate this e.g. recent EPYC processors run at up to 4800MT/s but a Ryzen 9 runs at up to 5200MT/s (without overclocking, with running at 6000MT/s isn't that uncommon) Zen3 EPYC has 8 channels per socket and uses the same 3200MT/s DDR4 as Zen3 desktops, so this little explains why desktops couldn't have e.g. 4 channels if there was a need for it. They already commonly have 4 DIMM slots. The rated speed for DDR5 is 8% higher for desktops, but the additional channels don't seem to be why. Intel uses 8 channels of DDR5 rather than 12 and it's still 4800MT/s. > so it's a benefit/cost balance calculation But that was my point?
HappyDaoDude 3y ago
It isn't so much the physical limits but more the cost of manufacturing a physically more complex board. This is why most low end GPU's typically have their memory bus width cut down to save on PCB and RAM complexity costs. An extreme example, the phone I am using has RAM running at 1.8Ghz but on a 16bit bus. Anything to keep costs down.

CoastalCoder 3y ago

I'm curious if anytime soon we'll start seeing optical interconnections between CPUs and on-device / same-chasis memory. I'm not sure what that would add for latency and cost, but I'm guessing that with optical wave guides, crosstalk isn't a concern. At least for the optical part of the data path.

kimixa 3y ago
The speed of light in an optical fiber is still only about 2*10^8 m/s - ddr5 6000's 3ghz clock means it still only travels 6.7cm/clock - so there'll still be difficulties around latency if it's not "close", and that's ignoring any issues of actually getting optical transceivers on the die itself.
- billti 3y ago
  It blew my mind when I learned that even if you could hand wire conceptually perfect switches and wires, once your board was over a few inches, the laws of physics (e.g., the speed of light) would mean you still couldn’t beat a modern CPU for clock speed. It really is insane the level at which current hardware works.
  - josephg 3y ago
    Yeah. Any time I wait for my computer to do something I think about this. In the 4 seconds discord took to open, my computer could do what, 5 billion operations per second per core? Even single threaded that’s 20 billion steps? And my internet connection could have delivered 600 million bytes of data - enough for a complete copy of windows xp. It’s wild.
    - steve1977 3y ago
      It’s wild and a sign how terribly inefficient most software is today
trealira 3y ago
> I'm curious if anytime soon we'll start seeing optical interconnections between CPUs and on-device / same-chasis memory. Isn't that kind of what those APUs sold by AMD are? And SoCs? I could be misunderstanding you, though.
ta988 3y ago
They are coming yes. They may not bring much in term of bandwidth per line, but will allow much more lines and entirely different geometries for the cpus allowing chips to be cooled on both sides etc.

AnthonyMouse 3y ago

> You have to recognize that each of those lines is creating a magnetic field, the closer you stack them, the more likely cross talk is. PC desktops have 2 memory channels. EPYC servers have 12 channels per socket. It's not the physics, it's that ~8-core processors generally aren't bounded by memory bandwidth and it's not worth the cost of increasing the number of pins. The server sockets do it because they have 128-core processors. If this changes what we may start seeing is CPUs (or iGPUs) with a few GB of HBM as L4 cache.

simcop2387 3y ago
Intel is doing this today in the Xeon MAX line, up to 64gb of hbm ram, you don't even necessarily need ddr5 on them. serve the home has a good article about them but it's all very new so it's hard to say how it'll shake out.
dathinab 3y ago
> not the physics no physics matter because it makes it much harder and in turn more costly to the more channels you have to have high quality reliable connections server RAM is for example also slower to compensate this e.g. recent EPYC processors run at up to 4800MT/s but a Ryzen 9 runs at up to 5200MT/s (without overclocking, with running at 6000MT/s isn't that uncommon) server also tend to always use EEC memory so it's a benefit/cost balance calculation
- AnthonyMouse 3y ago
  > server RAM is for example also slower to compensate this e.g. recent EPYC processors run at up to 4800MT/s but a Ryzen 9 runs at up to 5200MT/s (without overclocking, with running at 6000MT/s isn't that uncommon) Zen3 EPYC has 8 channels per socket and uses the same 3200MT/s DDR4 as Zen3 desktops, so this little explains why desktops couldn't have e.g. 4 channels if there was a need for it. They already commonly have 4 DIMM slots. The rated speed for DDR5 is 8% higher for desktops, but the additional channels don't seem to be why. Intel uses 8 channels of DDR5 rather than 12 and it's still 4800MT/s. > so it's a benefit/cost balance calculation But that was my point?

HappyDaoDude 3y ago

It isn't so much the physical limits but more the cost of manufacturing a physically more complex board. This is why most low end GPU's typically have their memory bus width cut down to save on PCB and RAM complexity costs. An extreme example, the phone I am using has RAM running at 1.8Ghz but on a 16bit bus. Anything to keep costs down.