3 ms·

Racing to idle is a very common power optimization technique

• vlovich123 6d ago
  Right, but HBM idle is significantly worse than LPDDR idle or even DDR idle for that matter. That matters a lot precisely because the device is idle most of the time. Your idle power draw dominates.
  • monster_truck 5d ago
    Comparing HBM to LPDDR is pretty silly. That's like a bath tub vs a urinal
    • imtringued 5d ago
      If LPDDR6 PIM ever becomes a thing then the advantage of HBM will shrink. I'm not saying LPDDR6 PIM will make HBM obsolete in the datacenter or where it is currently shining, I'm saying that large volumes have their own charm and it is more likely for LPDDR6 PIM to be in your laptop or smartphone or SBC than HBM. LPDDR6 PIM would primarily help the low end and mid range accelerator market. E.g. embedded models running on SBCs can be up to 1 GiB in size with acceptable performance, small models at 8 GiB become really easy to run at reasonable speeds on a smartphone and PIM enabled laptops or mini PCs make it possible to run 32 GiB models locally without compromise. Of course this also assumes that the associated accelerators (NPUs) will catch up too, but the general point is that you won't need a 5090 or a 4090 anymore.
    • vlovich123 5d ago
      This is literally a thread where people are saying “but why laptops no HBM”. It’s a perfectly cromulent response that meets the question where it is. It might be mildly a more defensible question for normal plugged in desktops but that would require a massive ecosystem change since people who build those traditionally really like their DIMMs and less buying a CPU that has non-swappable RAM built in, not to mention there’s no sustainable market for the higher cost and low volume product.