5 ms·

Nvidia Announces DGX GH200 AI Supercomputer

40 points by bfoks 3y ago

llm_nerd 3y ago
The picture almost looks like a 2U server, but it's actually 24 full server racks. https://www.anandtech.com/show/18877/nvidia-grace-hopper-has-entered-full-production-announcing-dgx-gh200-ai-supercomputer https://www.anandtech.com/show/18877/nvidia-grace-hopper-has...
abudabi123 3y ago
Certainly doesn't look as impressive as Tesla's Exapod housing. How does the first principles engineering rate comparing Tesla's Exapod to this?
- theaiquestion 3y ago
  Well if you want to compare them, Nvidia's has over 10x the memory and has historically sold > 1 units of their HPC products outside their company. I'm not sure the aesthetic of what are essentially cabinets in a warehouse is that much of a consideration with HPC?
  - PeterStuer 3y ago
    If you are convincing the board to spend that sort of cash you want something photogenic.
  - fnordpiglet 3y ago
    Most super computers - at least the ones I’ve visited - have some nice chrome. Their output tends to be a couple of numbers or something equally uninspiring so making them look awesome is key to funding.
- shaklee3 3y ago
  Tesla's exapod doesn't and won't exist for the public outside of press releases, for one.
ChatGTP 3y ago
“Generative AI, large language models and recommender systems are the digital engines of the modern economy,” said Jensen Huang, founder and CEO of NVIDIA. Are they really?
qwertox 3y ago
The amount of RAM, even if it's just DIMM, made my jaw drop. I wish I could fast forward time for 10 years to see how integrated and affordable this card has become, or if it would even be possible to integrate something like this into a smartphone in 20 years. Just imagine how people would be interacting with their phones then, interacting with them as if they were humans. No other technology gets me as excited as storage and compute, I guess this is how car people get excited if a new Maserati or something else "with numbers" is launched.
- PeterStuer 3y ago
  Most of that memory would be LPDDR5X. https://semiconductor.samsung.com/dram/lpddr/lpddr5x/ https://semiconductor.samsung.com/dram/lpddr/lpddr5x/
  - qwertox 3y ago
    Oh, right, I misnamed DDR.
- jiggawatts 3y ago
  Assuming Moore's law continues, the answer is probably yes. At some point, you'll have more computing power in your pocket. For example, "ASCI Red" was the #1 in the TOP500 supercomputers list in the year 2000 with about 2 teraflops... roughly the same[1] as an iPhone Pro 14. The #1 from June 2003 -- almost exactly two decades ago -- is about the same speed as a modern NVIDIA desktop GPU. As recently as 2014 there were still "supercomputers" near the bottom of the TOP500 list that are about equivalent to 2x RTX 4090 GPUs. That's a rare but readily available configuration for a gaming enthusiast. So it's about 20-25 years until a supercomputer fits in your pocket, and as little as a decade until you can have one under your desk heating your room. [1] Most supercomputers measure 64-bit ops, whereas these days people focus on 32-bit ops, but whatever. It's just a factor of two.
- ksec 3y ago
  >or if it would even be possible to integrate something like this into a smartphone in 20 years. I am going to answer this as probably not. If you assume we still get 2X density improvement every 2 years ( which we dont ) and the density shrink applies to every type of logic and SRAM ( which we dont ) and we dont slow down the pace for 20 years ( which we dont as we are already slowing ) and we dont hit any heat limitation ( which we do ) and we dont hit any other fundamental limit of law of physics ( which we will, and get quantum tunnel effect ) etc etc There are 24 Racks of Computer there. Even if you ignore the I/O and Power. There are 16 Racks of Raw computing. Over 20 years, 10 iteration, you get 1024x increase in density. 16 Racks, or 640U Server, you get 0.625 U by 2043. Half a Server, hardly the size of Smartphone.
pjmlp 3y ago
For anyone that cares, DGX OS is based on Ubuntu, and there are packages for Red Hat Enterprise Linux as alternative. https://docs.nvidia.com/base-os/index.html https://docs.nvidia.com/base-os/index.html
louissan 3y ago
price? 8-}

40 points by bfoks 3y ago

The picture almost looks like a 2U server, but it's actually 24 full server racks. https://www.anandtech.com/show/18877/nvidia-grace-hopper-has-entered-full-production-announcing-dgx-gh200-ai-supercomputer https://www.anandtech.com/show/18877/nvidia-grace-hopper-has...

abudabi123 3y ago

Certainly doesn't look as impressive as Tesla's Exapod housing. How does the first principles engineering rate comparing Tesla's Exapod to this?

theaiquestion 3y ago
Well if you want to compare them, Nvidia's has over 10x the memory and has historically sold > 1 units of their HPC products outside their company. I'm not sure the aesthetic of what are essentially cabinets in a warehouse is that much of a consideration with HPC?
- PeterStuer 3y ago
  If you are convincing the board to spend that sort of cash you want something photogenic.
- fnordpiglet 3y ago
  Most super computers - at least the ones I’ve visited - have some nice chrome. Their output tends to be a couple of numbers or something equally uninspiring so making them look awesome is key to funding.
shaklee3 3y ago
Tesla's exapod doesn't and won't exist for the public outside of press releases, for one.

ChatGTP 3y ago

“Generative AI, large language models and recommender systems are the digital engines of the modern economy,” said Jensen Huang, founder and CEO of NVIDIA. Are they really?

qwertox 3y ago

The amount of RAM, even if it's just DIMM, made my jaw drop. I wish I could fast forward time for 10 years to see how integrated and affordable this card has become, or if it would even be possible to integrate something like this into a smartphone in 20 years. Just imagine how people would be interacting with their phones then, interacting with them as if they were humans. No other technology gets me as excited as storage and compute, I guess this is how car people get excited if a new Maserati or something else "with numbers" is launched.

PeterStuer 3y ago
Most of that memory would be LPDDR5X. https://semiconductor.samsung.com/dram/lpddr/lpddr5x/ https://semiconductor.samsung.com/dram/lpddr/lpddr5x/
- qwertox 3y ago
  Oh, right, I misnamed DDR.
jiggawatts 3y ago
Assuming Moore's law continues, the answer is probably yes. At some point, you'll have more computing power in your pocket. For example, "ASCI Red" was the #1 in the TOP500 supercomputers list in the year 2000 with about 2 teraflops... roughly the same[1] as an iPhone Pro 14. The #1 from June 2003 -- almost exactly two decades ago -- is about the same speed as a modern NVIDIA desktop GPU. As recently as 2014 there were still "supercomputers" near the bottom of the TOP500 list that are about equivalent to 2x RTX 4090 GPUs. That's a rare but readily available configuration for a gaming enthusiast. So it's about 20-25 years until a supercomputer fits in your pocket, and as little as a decade until you can have one under your desk heating your room. [1] Most supercomputers measure 64-bit ops, whereas these days people focus on 32-bit ops, but whatever. It's just a factor of two.
ksec 3y ago
>or if it would even be possible to integrate something like this into a smartphone in 20 years. I am going to answer this as probably not. If you assume we still get 2X density improvement every 2 years ( which we dont ) and the density shrink applies to every type of logic and SRAM ( which we dont ) and we dont slow down the pace for 20 years ( which we dont as we are already slowing ) and we dont hit any heat limitation ( which we do ) and we dont hit any other fundamental limit of law of physics ( which we will, and get quantum tunnel effect ) etc etc There are 24 Racks of Computer there. Even if you ignore the I/O and Power. There are 16 Racks of Raw computing. Over 20 years, 10 iteration, you get 1024x increase in density. 16 Racks, or 640U Server, you get 0.625 U by 2043. Half a Server, hardly the size of Smartphone.

pjmlp 3y ago

For anyone that cares, DGX OS is based on Ubuntu, and there are packages for Red Hat Enterprise Linux as alternative. https://docs.nvidia.com/base-os/index.html https://docs.nvidia.com/base-os/index.html

louissan 3y ago

price? 8-}