3 ms·

I have heard these are 100k each... anyone know if that's correct? Guessing that's list, and no one pays list, but still...

finexplained 3y ago
I thought the H100s were ~30-40k each. But they're not widely available and you usually buy multiple boxes from vendors that also come with expensive CPUs/RAM etc.
- aiappreciator 3y ago
  H100s are intended to be retailed piecemeal. The big enterprise model is DGX GH200, at $10 mil each. Its just that current supply is extremely short, so the H100s end up only available to big buyers. But that will be resolved in time. Nvidia wants every university lab to have a H100 so no competitor sneaks in there.
tikkun 3y ago
I'm writing a cool mega post on this at the moment – not published yet but here's the excerpt on pricing: How much do these GPUs cost? H100s are around $30-33k at the IT hardware resellers CDW and SHI (https://www.cdw.com/product/nvidia-h100-gpu-computing-processor-nvidia-h100-tensor-core-80-gb/7367181 https://www.cdw.com/product/nvidia-h100-gpu-computing-proces..., https://www.shi.com/product/45671009/NVIDIA-H100-GPU-computing-processor https://www.shi.com/product/45671009/NVIDIA-H100-GPU-computi...) Supermicro’s HGX H100 8x GPU server is $297k at the reseller Dihuni (https://www.dihuni.com/product/supermicro-8125gs-tnhr-server/ https://www.dihuni.com/product/supermicro-8125gs-tnhr-server...) DGX H100 is $521k at the reseller Insight (https://www.insight.com/en_US/shop/product/DGXH-G640F+P2CMI60/NVIDIA/DGXH-G640F+P2CMI60/NVIDIA-DGX-H100-P4387-SYSTEM-640GB-FULL-STD-SUPPORT-5-YRS/ https://www.insight.com/en_US/shop/product/DGXH-G640F+P2CMI6...) The DGX GH200 might cost in the range of $10mm-20mm or more (A guesstimate ballpark from an exec at a cloud company I talked with) If anyone wants to pre-review the post and can offer thoughtful comments, my email's in my profile. And to clarify the difference between all of these product names, I put together this diagram - https://gpus.llm-utils.org/dgx-gh200-vs-gh200-vs-h100/ https://gpus.llm-utils.org/dgx-gh200-vs-gh200-vs-h100/. I don't have the HGX H100 or the DGX H100 on there though - but the HGX H100 is a reference platform for OEMs to design and make H100 based servers with either 4x H100s or 8x H100s (https://nvdam.widen.net/s/5kgbjq2v2t/hpc-hgx-h100-datasheet-nvidia-web https://nvdam.widen.net/s/5kgbjq2v2t/hpc-hgx-h100-datasheet-...) and the DGX H100 is the official Nvidia server with 8x H100s (https://resources.nvidia.com/en-us-dgx-systems/ai-enterprise-dgx https://resources.nvidia.com/en-us-dgx-systems/ai-enterprise...).
- mrcggl 3y ago
  I don't get how Lambda makes money with those prices. 2$ per hour seems too cheap for a >30k$ GPU
  - huac 3y ago
    it's marketing :)
huac 3y ago
from somebody who bought 2500 of them: https://news.ycombinator.com/item?id=36313960 https://news.ycombinator.com/item?id=36313960

finexplained 3y ago

I thought the H100s were ~30-40k each. But they're not widely available and you usually buy multiple boxes from vendors that also come with expensive CPUs/RAM etc.

aiappreciator 3y ago
H100s are intended to be retailed piecemeal. The big enterprise model is DGX GH200, at $10 mil each. Its just that current supply is extremely short, so the H100s end up only available to big buyers. But that will be resolved in time. Nvidia wants every university lab to have a H100 so no competitor sneaks in there.

tikkun 3y ago

I'm writing a cool mega post on this at the moment – not published yet but here's the excerpt on pricing: How much do these GPUs cost? H100s are around $30-33k at the IT hardware resellers CDW and SHI (https://www.cdw.com/product/nvidia-h100-gpu-computing-processor-nvidia-h100-tensor-core-80-gb/7367181 https://www.cdw.com/product/nvidia-h100-gpu-computing-proces..., https://www.shi.com/product/45671009/NVIDIA-H100-GPU-computing-processor https://www.shi.com/product/45671009/NVIDIA-H100-GPU-computi...) Supermicro’s HGX H100 8x GPU server is $297k at the reseller Dihuni (https://www.dihuni.com/product/supermicro-8125gs-tnhr-server/ https://www.dihuni.com/product/supermicro-8125gs-tnhr-server...) DGX H100 is $521k at the reseller Insight (https://www.insight.com/en_US/shop/product/DGXH-G640F+P2CMI60/NVIDIA/DGXH-G640F+P2CMI60/NVIDIA-DGX-H100-P4387-SYSTEM-640GB-FULL-STD-SUPPORT-5-YRS/ https://www.insight.com/en_US/shop/product/DGXH-G640F+P2CMI6...) The DGX GH200 might cost in the range of $10mm-20mm or more (A guesstimate ballpark from an exec at a cloud company I talked with) If anyone wants to pre-review the post and can offer thoughtful comments, my email's in my profile. And to clarify the difference between all of these product names, I put together this diagram - https://gpus.llm-utils.org/dgx-gh200-vs-gh200-vs-h100/ https://gpus.llm-utils.org/dgx-gh200-vs-gh200-vs-h100/. I don't have the HGX H100 or the DGX H100 on there though - but the HGX H100 is a reference platform for OEMs to design and make H100 based servers with either 4x H100s or 8x H100s (https://nvdam.widen.net/s/5kgbjq2v2t/hpc-hgx-h100-datasheet-nvidia-web https://nvdam.widen.net/s/5kgbjq2v2t/hpc-hgx-h100-datasheet-...) and the DGX H100 is the official Nvidia server with 8x H100s (https://resources.nvidia.com/en-us-dgx-systems/ai-enterprise-dgx https://resources.nvidia.com/en-us-dgx-systems/ai-enterprise...).

mrcggl 3y ago
I don't get how Lambda makes money with those prices. 2$ per hour seems too cheap for a >30k$ GPU
- huac 3y ago
  it's marketing :)

huac 3y ago

from somebody who bought 2500 of them: https://news.ycombinator.com/item?id=36313960 https://news.ycombinator.com/item?id=36313960