3 ms·

Going off of how the 4090 and 7900 xtx is arranged I think you could maybe fit on or two chips more around the die over their 12, but that's still a far cry fro

Going off of how the 4090 and 7900 xtx is arranged I think you could maybe fit on or two chips more around the die over their 12, but that's still a far cry from 128. That would probably just need a shared bus like normal DDR as you're not fitting that much with 16 gbit density

SmellTheGlove 2y ago
What if we did what others suggested was the practical limit - 48GB. Then just put 2-3 cards in the system and maybe had a little bridge over a separate bus for them to communicate?
- deleted 2y ago
  [deleted]
- Numerlor 2y ago
  I believe that would need some software work from Intel where they're lacking a bit now with their delayed start. Not sure how the frameworks themselves split up the inference work to avoid crossing GPUs as the bandwidth is horrible there. If we're being reasonable and say that you're not using a modern HEDT CPU that costs a couple thousand, the best a consumer botherboard can get right now would be 2x 8x PCIe gen 5 at 32GB/s and one chipset x8 PCIe gen 4 at 16GB/s. I'm not sure if a motherboard like that actually exists but Intel's chipset should allow it; AMD only does x4 to chipset so the third slot is limited by that
ryao 2y ago
Look at the 3090, which uses 24 chips (12 on one side and 12 on another). Pushing it to 32 is doable. 32 is all you need to reach 128GB VRAM with the 32Gbit GDDR7 chips that should be on the market in the near future.
- Numerlor 2y ago
  Where would you route the connection to the additional 4 groups of chips around the die? The PCIe connection needs to be there too, and they also may not like power delivery going through them
  - ryao 2y ago
    Nvidia has done a 512-bit bus in the past. The 3090 has 4 groups of 3 on each side of the card. Switching to 4 groups of 4 should be doable visually. That said, I would not want to be the one responsible for doing the trace routing.

SmellTheGlove 2y ago

What if we did what others suggested was the practical limit - 48GB. Then just put 2-3 cards in the system and maybe had a little bridge over a separate bus for them to communicate?

deleted 2y ago
[deleted]
Numerlor 2y ago
I believe that would need some software work from Intel where they're lacking a bit now with their delayed start. Not sure how the frameworks themselves split up the inference work to avoid crossing GPUs as the bandwidth is horrible there. If we're being reasonable and say that you're not using a modern HEDT CPU that costs a couple thousand, the best a consumer botherboard can get right now would be 2x 8x PCIe gen 5 at 32GB/s and one chipset x8 PCIe gen 4 at 16GB/s. I'm not sure if a motherboard like that actually exists but Intel's chipset should allow it; AMD only does x4 to chipset so the third slot is limited by that

ryao 2y ago

Look at the 3090, which uses 24 chips (12 on one side and 12 on another). Pushing it to 32 is doable. 32 is all you need to reach 128GB VRAM with the 32Gbit GDDR7 chips that should be on the market in the near future.

Numerlor 2y ago
Where would you route the connection to the additional 4 groups of chips around the die? The PCIe connection needs to be there too, and they also may not like power delivery going through them
- ryao 2y ago
  Nvidia has done a 512-bit bus in the past. The 3090 has 4 groups of 3 on each side of the card. Switching to 4 groups of 4 should be doable visually. That said, I would not want to be the one responsible for doing the trace routing.