3 ms·

I don't fully understand the source of the "total bytes" constraint, but a major factor may be because HBM4 / HBM4E can only make use of the footprint directly

I don't fully understand the source of the "total bytes" constraint, but a major factor may be because HBM4 / HBM4E can only make use of the footprint directly above the processor/logic die (or in direct vicinity of its interconnect), while traditional DRAM can be placed further away where there's lots of real estate on the motherboard. I gather a practical max ceiling today is a stack of 16 chips in height yielding 64GB? These chips have a massive bus size of 2048 bits, instead of the 64 or 128 bits (dual channel) used by DDR5. That's what gives them their order-of-magnitude bandwidth speedup. But even though they technically pack in more capacity per square millimeter of motherboard, I gather they take up more space than older technologies once you account for the vias and interconnects to route all those signals.

threecheese 13d ago
Thanks for that, just went down an interesting rabbit hole. Many of us were hoping this re-tooling would eventually trickle some fast RAM down to DRAM-exhausted PCs, but given it would require a rearchitecture of the motherboard it's unlikely.
- sroussey 13d ago
  HBM also trades bandwidth for latency, and your regular computing is much more sensitive to latency than bandwidth.
- craigjb 13d ago
  HBM4 has over 2048 signals to the processor’s PHY with tight signal integrity requirements that require the HBM stack to be < 0.5 mm from the processor die. That’s why HBM integration is done with interposers (soldered on the package). So, it’d be the CPU package that integrates it. Motherboard is too far away.
  - Melatonic 13d ago
    Kind of seems like we should be making chips with both. Big HBM stack on top as a sort of huge L5 cache like thing. And then a bunch of DRAM type sockets (like LPCAMM) around the exterior.
    - craigjb 12d ago
      For chips with integrated CPU+GPU+NPU, it could be worth it tech-wise. The GPU and NPU can eat HBM bandwidth. For general purpose CPU code, the HBM would likely not be worth it. It's high bandwidth, but you trade latency, and general CPU code is branchy. Economics-wise, the HBM stacks alone will cost more than a consumer CPU (or APU). [edit] The packaging needed to support HBM is also much more expensive too. If demand for current HBM applications tanks and the manufacturing lines need filled, then maybe. Currently, the price point would make it very very niche.

threecheese 13d ago

Thanks for that, just went down an interesting rabbit hole. Many of us were hoping this re-tooling would eventually trickle some fast RAM down to DRAM-exhausted PCs, but given it would require a rearchitecture of the motherboard it's unlikely.

sroussey 13d ago
HBM also trades bandwidth for latency, and your regular computing is much more sensitive to latency than bandwidth.
craigjb 13d ago
HBM4 has over 2048 signals to the processor’s PHY with tight signal integrity requirements that require the HBM stack to be < 0.5 mm from the processor die. That’s why HBM integration is done with interposers (soldered on the package). So, it’d be the CPU package that integrates it. Motherboard is too far away.
- Melatonic 13d ago
  Kind of seems like we should be making chips with both. Big HBM stack on top as a sort of huge L5 cache like thing. And then a bunch of DRAM type sockets (like LPCAMM) around the exterior.
  - craigjb 12d ago
    For chips with integrated CPU+GPU+NPU, it could be worth it tech-wise. The GPU and NPU can eat HBM bandwidth. For general purpose CPU code, the HBM would likely not be worth it. It's high bandwidth, but you trade latency, and general CPU code is branchy. Economics-wise, the HBM stacks alone will cost more than a consumer CPU (or APU). [edit] The packaging needed to support HBM is also much more expensive too. If demand for current HBM applications tanks and the manufacturing lines need filled, then maybe. Currently, the price point would make it very very niche.