Every modern motherboard is dual-channel DDR5 at a minimum, maybe quad-channel. 2x sticks of 32GB/s RAM, properly configured, will run at 64GB/s of bandwidth.

Every modern motherboard is dual-channel DDR5 at a minimum, maybe quad-channel. 2x sticks of 32GB/s RAM, properly configured, will run at 64GB/s of bandwidth. Modern servers are quad, hex, or oct-channel (4x, 6x, or 8x parallel sticks of RAM) in practice. Or even more (ex: Intel Xeon Platinums are 6x channel per socket, so an 8x socket 8x CPU Xeon Platinum will be like 48x DDR5 parallel RAM sticks). ---------- PCIe x16 by the way, is 16x parallel lanes of PCIe. All the parallelism is already innate in modern systems. --------- L3 cache is TB/s bandwidth IIRC. CPUs inside of CPU-space will automatically be caching a lot of those RAM commands, so you can go above the RAM-bandwidth limitations in practice, though it depends on how your code accesses RAM. GPUs have very small caches, and have higher latency to those caches. Instead, GPUs rely upon register-space and extreme amounts of SMT/wavefronts to kinda-sorta hyperthread their cores to hide all that latency.