3 ms·

Benchmarks were just run, MI300x is onpar/better than an H100. Next generation of MI (MI325x) is coming out end of the year and those specs look fantastic too.

Benchmarks were just run, MI300x is onpar/better than an H100. Next generation of MI (MI325x) is coming out end of the year and those specs look fantastic too. Especially on the all important memory front. 288GB is fantastic. Both companies will leapfrog each other with new releases. Anyone who believes that there should only be a single vendor for all AI compute will quickly find themselves on the wrong side of history

nabla9 2y ago
Comparisons against H100 I have seen are always: 8x AMD MI300X (192GB, 750W) 8x H100 SXM5 (80GB, 700W) Never against 8x H100 NVL (188GB, <800W) What the customer does not see is how AMD must spend 2 times more money to produce a chip that is competitive against architecture that is soon 2 years old.
- latchkey 2y ago
  > Never against 8x H100 NVL (188GB, <800W) Probably because they aren't widely available yet. It is also a dual card to get that much memory, which is still less than 192GB and far less than 288GB. https://www.anandtech.com/show/18780/nvidia-announces-h100-nvl-max-memory-server-card-for-large-language-models https://www.anandtech.com/show/18780/nvidia-announces-h100-n... > What the customer does not see is how AMD must spend 8-10 times more money to produce a chip that is competitive against architecture that is soon 2 years old. Source?
- claytonjy 2y ago
  as sibling mentioned the 188GB is for a pair. The memory bump is from enabling the 6th block of memory that is otherwise disabled on H100s. I assume an "NVL box" is still 8 total GPUs, so more like 8x H100 NVL (94GB, 800W) the AMD box has a lot more GPU memory
talldayo 2y ago
> 288GB is fantastic This reminds me of those "192GB is fantastic" people that bought maxed-out M2 Ultras for AI inference. It can be awesome, but you need a substantial amount of interconnect bandwidth and powerful enough local compute before it's competitive. In products where AI is an afterthought, you're fighting against much different constraints than just having a lot of high-bandwidth memory. I've always rooted for Team Red when they made an effort to do things open-source and transparently. They're a good role-model for the rest of the industry, in a certain sense. But I have to make peace with the fact that client-side AI running on my AMD machines isn't happening. Meanwhile, I've been using CUDA, CUDNN, CUBLAS, DLSS, on my Nvidia machine for years. On Linux!
- latchkey 2y ago
  This response feels like you could be conflating desktop usage with enterprise compute?

nabla9 2y ago

Comparisons against H100 I have seen are always: 8x AMD MI300X (192GB, 750W) 8x H100 SXM5 (80GB, 700W) Never against 8x H100 NVL (188GB, <800W) What the customer does not see is how AMD must spend 2 times more money to produce a chip that is competitive against architecture that is soon 2 years old.

latchkey 2y ago
> Never against 8x H100 NVL (188GB, <800W) Probably because they aren't widely available yet. It is also a dual card to get that much memory, which is still less than 192GB and far less than 288GB. https://www.anandtech.com/show/18780/nvidia-announces-h100-nvl-max-memory-server-card-for-large-language-models https://www.anandtech.com/show/18780/nvidia-announces-h100-n... > What the customer does not see is how AMD must spend 8-10 times more money to produce a chip that is competitive against architecture that is soon 2 years old. Source?
claytonjy 2y ago
as sibling mentioned the 188GB is for a pair. The memory bump is from enabling the 6th block of memory that is otherwise disabled on H100s. I assume an "NVL box" is still 8 total GPUs, so more like 8x H100 NVL (94GB, 800W) the AMD box has a lot more GPU memory

talldayo 2y ago

> 288GB is fantastic This reminds me of those "192GB is fantastic" people that bought maxed-out M2 Ultras for AI inference. It can be awesome, but you need a substantial amount of interconnect bandwidth and powerful enough local compute before it's competitive. In products where AI is an afterthought, you're fighting against much different constraints than just having a lot of high-bandwidth memory. I've always rooted for Team Red when they made an effort to do things open-source and transparently. They're a good role-model for the rest of the industry, in a certain sense. But I have to make peace with the fact that client-side AI running on my AMD machines isn't happening. Meanwhile, I've been using CUDA, CUDNN, CUBLAS, DLSS, on my Nvidia machine for years. On Linux!

latchkey 2y ago
This response feels like you could be conflating desktop usage with enterprise compute?