2 ms·

Bandwidth depends very much on on bus width. If its fast LPDDR5x (9600 MT/s) with 512 bit bus width (8 64bit channels (actually multiples of quad 16 bit subcha

• jychang 1y ago
  1. 600GB/sec is still slow as hell. You might as well as use regular DDR5 RAM then if you're so slow, you can spec regular system DDR5 RAM faster than 600GB/sec. The half decade old consumer 3090 is 1.5x that speed. The current 5090 is 1,792 GB/s. The current nvidia datacenter cards are 8 TB/s. What's the point of having lots of VRAM if your system RAM is faster? For context: if you have a 160GB dense ML model in VRAM and you're just running 600GB/sec, you can do... roughly 4 tokens per second AT BEST. That massive amount of VRAM is unusable if it's slow. 2. 512 bit LPDDR5x is most likely just 512GB/sec with typical LPDDR5x that's not overly expensive. I would be HIGHLY surprised if they gave it the more expensive RAM that'd break 600GB/sec. The Intel B60 is at 456 GB/s and that's using GDDR6. Honestly, you're better off waiting for regular DDR6 to come out in a year and just build a system using that.