2 ms·

GDDR isn't as special as you think, and it's special in exactly the opposite way that HBM is special. HBM achieves high bandwidth by using extremely large bus w

GDDR isn't as special as you think, and it's special in exactly the opposite way that HBM is special. HBM achieves high bandwidth by using extremely large bus widths running at fairly slow speeds per pin. GDDR achieves high bandwidth by running at very high speeds per pin and using medium to large bus widths. Desktop CPUs using DDR5 have low memory bandwidth because they have narrow bus widths (128bit total) at low speeds. Server CPUs use the same memory at the same per-pin speeds but with much larger bus widths to achieve total bandwidth comparable to a mid-range GPU but supporting orders of magnitude more memory capacity. Apple's high-end chips likewise uses significantly larger bus widths than desktop CPUs, combined with slightly per-pin speeds (LPDDR rather than DDR) to also reach GPU-like total bandwidth. I don't think it's accurate to single out any one of the above technologies as being the "normal" DRAM. GDDR, LPDDR, and DDR are all fairly different from each other and all quite mainstream.

LoganDark 3y ago
GDDR is "special" because it's far closer to the GPU die. System memory isn't that close to the CPU, so any integrated GPU won't be able to make as good use of it as a dedicated GPU with co-located VRAM.
- wtallis 3y ago
  > GDDR is "special" because it's far closer to the GPU die. LPDDR is always at least as close to the processor, and is often in-package so significantly closer. HBM is always in-package. But all of that is a red herring, because the distance is only loosely correlated with performance and is a completely irrelevant metric if you've already accounted for bus width and clock rate.
  - LoganDark 3y ago
    > LPDDR is always at least as close to the processor, and is often in-package Are you talking about SoCs? Regular laptop/desktop APUs don't usually include their own RAM.
    - wtallis 3y ago
      It would be pretty weird to ignore the existence of smartphones when discussing LPDDR, since they probably account for most of the volume of LPDDR sold. Lots of smartphone SoCs, all of Apple's laptop SoCs, and that one Intel custom job for Asus last year have LPDDR in-package, while the rest of the x86 systems with LPDDR have it on the motherboard at similar distances from the processor as GDDR.
      - LoganDark 3y ago
        > It would be pretty weird to ignore the existence of smartphones Not really. It's pretty difficult to do LLM development work on a smartphone.

LoganDark 3y ago

GDDR is "special" because it's far closer to the GPU die. System memory isn't that close to the CPU, so any integrated GPU won't be able to make as good use of it as a dedicated GPU with co-located VRAM.

wtallis 3y ago
> GDDR is "special" because it's far closer to the GPU die. LPDDR is always at least as close to the processor, and is often in-package so significantly closer. HBM is always in-package. But all of that is a red herring, because the distance is only loosely correlated with performance and is a completely irrelevant metric if you've already accounted for bus width and clock rate.
- LoganDark 3y ago
  > LPDDR is always at least as close to the processor, and is often in-package Are you talking about SoCs? Regular laptop/desktop APUs don't usually include their own RAM.
  - wtallis 3y ago
    It would be pretty weird to ignore the existence of smartphones when discussing LPDDR, since they probably account for most of the volume of LPDDR sold. Lots of smartphone SoCs, all of Apple's laptop SoCs, and that one Intel custom job for Asus last year have LPDDR in-package, while the rest of the x86 systems with LPDDR have it on the motherboard at similar distances from the processor as GDDR.
    - LoganDark 3y ago
      > It would be pretty weird to ignore the existence of smartphones Not really. It's pretty difficult to do LLM development work on a smartphone.