Putting a really big DRAM cache right next to the CPU would not actually help all that much. The reason DRAM is so slow is because selecting, sensing and amplif

Putting a really big DRAM cache right next to the CPU would not actually help all that much. The reason DRAM is so slow is because selecting, sensing and amplifying is slow, not because it's far away. It's still so far away just because adding 30% bus latency doesn't sting enough to bring it closer. If you want to reduce latency, what you want is SRAM (expensive), or one of the experimental memory technologies that scales to SRAM-like latencies. The most immediate candidate is T-Ram, which has access latency of <2x 6T-SRAM, is nearly as dense as DRAM, and is presently on the roadmap for mass production for the 32/22nm GlobalFoundries processes. Putting DRAM on-package or on-die would give nice boosts to bandwidth. It could really help integrated graphics.