High bandwidth GDDR5 is more expensive than DDR4. HBM is more expensive again. On top of that the high bandwidth chips are lower density, and the complexity o

High bandwidth GDDR5 is more expensive than DDR4. HBM is more expensive again. On top of that the high bandwidth chips are lower density, and the complexity of higher bandwidth links gets more complicated as you get more chips involved. Low end servers use normal dimm, but typically not many dimms per channel, typically 1-3. But to hit the higher memory configurations you end up putting buffer chips on the dimms which makes them more expensive and also slightly higher latency. So 500GB/sec to 128GB ram using HBM2 would be difficult, expensive, and likely wouldn't fit in today's GPU form factors... yet. GPUs generally seem to be mostly hungry for more bandwidth and not that many cases (percentage wise) is 16GB a major limitation. However in those cases check out the power9, the nvlink that x86's uses for GPU<->GPU communication can be used for CPU <-> GPU communication and avoid the high latency/low bandwidth pci-e links.