2 ms·
Because you can't stack that much ram on a GPU without sufficient channels to do so. You could probably do 64GB on GDDR6 but you can't do 128GB on GDDR6 without
by Sparkyte 2y ago
Because you can't stack that much ram on a GPU without sufficient channels to do so. You could probably do 64GB on GDDR6 but you can't do 128GB on GDDR6 without more memory channels. 2GB per chip per channel is the current limit for GDDR6 this is why HBM was invented.
It is why you can only see GPUs with 24GB of memory at the moment.
HBM2 can handle 64GB ( 4 x 8GB Stack ) ( Total capacity 128GB )
HBM3 can handle 192GB ( 4 x 24GB Stack ) ( Total capacity 384GB )
You can not do this with GDDR6.
- ryao 2y agoLook at the RTX 3090 and RTX A6000 (the non-Ada one). They both have 24 memory chips with a 384-bit memory bus, but one has 24GB of VRAM and the other has 48GB of VRAM. They both have two chips per channel. This breaks the 24GB VRAM limit that you claim to exist.
- Sparkyte 2y agoNope it doesn't that is 12 channels 2 modules per channel each module at 2GB is the largest the chip gets and the cost of those drastically increase the cost. The problem you experience with GDDR6 and channels is width. You get higher in size but you'll never be able to fill the memory fast enough to be cost effective. Also why HBM exists. Lets say the top is 960GB/s for an A6000 for GPU memory. HBM3 is 3.35 TB/s. If Intel wanted to make better GPUs it needs to switch to HBM.
- ryao 2y ago> Nope it doesn't that is 12 channels 2 modules per channel each module at 2GB You say nope and then proceed to say exactly what I said. When if the ICs are limited to 16Gbit density for now, the 24GB limit you mentioned was wrong, since you assumed single rank was the highest possible. As for HBM, that is earmarked for their gaudi 3 line. There is no chance of it being put into a consumer product, as it would be like selling gold at pyrite pricing.