4 ms·
Comparisons against H100 I have seen are always: 8x AMD MI300X (192GB, 750W) 8x H100 SXM5 (80GB, 700W) Never against 8x H100 NVL (188GB, <800W) What
by nabla9 2y ago
Comparisons against H100 I have seen are always:
8x AMD MI300X (192GB, 750W)
8x H100 SXM5 (80GB, 700W)
Never against 8x H100 NVL (188GB, <800W)
What the customer does not see is how AMD must spend 2 times more money to produce a chip that is competitive against architecture that is soon 2 years old.
- latchkey 2y ago> Never against 8x H100 NVL (188GB, <800W) Probably because they aren't widely available yet. It is also a dual card to get that much memory, which is still less than 192GB and far less than 288GB. https://www.anandtech.com/show/18780/nvidia-announces-h100-nvl-max-memory-server-card-for-large-language-models https://www.anandtech.com/show/18780/nvidia-announces-h100-n... > What the customer does not see is how AMD must spend 8-10 times more money to produce a chip that is competitive against architecture that is soon 2 years old. Source?
- claytonjy 2y agoas sibling mentioned the 188GB is for a pair. The memory bump is from enabling the 6th block of memory that is otherwise disabled on H100s. I assume an "NVL box" is still 8 total GPUs, so more like 8x H100 NVL (94GB, 800W) the AMD box has a lot more GPU memory