4 ms·

Nvidia reveals Blackwell B200 GPU, the 'most powerful chip' for AI

24 points by mfiguiere 3y ago

reaperman 3y ago
The article makes no mention at all of how much VRAM they might have. Other articles state plans for up to 192GB of HMB3e, and a power draw of 1000 watts.
- deleted 3y ago
  [deleted]
klysm 3y ago
It's terrifying to think about how many watts of power are being spent on generating useless trash
- epistasis 3y ago
  This is a bit vague, are you talking about this new GPU/LLMs, social media, industrial society, or something else?
- vasco 3y ago
  Most human consumption is useless trash by some metric.
- reaperman 3y ago
  Fun fact: each human uses an average 100W of power and generates mostly useless crap.
  - visarga 3y ago
    > each human uses an average 100W of power Does that include heating, electricity and fuel spent for the benefit of the human? How about infrastructure? Externalities are huge.
deleted 3y ago
[deleted]
epistasis 3y ago
> next-gen NVLink switch that lets 576 GPUs talk to each other, with 1.8 terabytes per second of bidirectional bandwidth. Well that's a pretty big step up from the 18 for the H100...
sp332 3y ago
I know it’s only FP4, but 1.4 exaflops in one rack is still crazy.
- alecco 3y ago
  With scaling and some tricks, FP4 inference can be very close to 16 bit. And most software out there is still missing on FP8 lol
  - cyanydeez 3y ago
    Doesn't that suggest the training method is flawed?
    - sp332 3y ago
      For training, you need to follow a gradient, and usually the gradient is small enough that you need the precision. Training in 8-bit is fairly new but Google and now Grok seem to have gotten good results.
      - alecco 3y ago
        And Microsoft.
      - cyanydeez 3y ago
        So these cards are geared towards inference ? There's a register article showing amds new instinct accelerators have much better.FP64
        sp332 3y ago
        The article has numbers for both training, I think in FP8, and inference in FP4.
  - deleted 3y ago
    [deleted]
jpeggtulsa 3y ago
[dead]
- _yo2u 3y ago
  you mean the chip that only has 44GB of memory on one chip and doesn't mention how fast chips can talk between each other?
  - jpeggtulsa 3y ago
    Why cut up a a wafer of chips, package each with HBM, put the package on a board, connect to CPUs with a fabric, then tie them all back together with networking chips and cables? Their 3 new clusters are the top 3 biggest AI training platforms in the world. Comparing the WSE-3 against Nvidia's H100. "It's got 52 times more cores. It's got 800 times more memory on chip. It's got 7,000 times more memory bandwidth and more than 3,700 times more fabric bandwidth. But they don't sell the chips, they build the clusters and sell the compute power. Except for a couple they built in Dubai. Plus, newer models are moving from Transformer to Mamba and don't need as much memory, because they save the important information, not everything it's been trained on.
shouche 3y ago
Explained by AI https://xplained.vercel.app/ask?q=Nvidia%20Blackwell https://xplained.vercel.app/ask?q=Nvidia%20Blackwell Additional details on vRAM and spec: https://xplained.vercel.app/ask?q=Nvidia%20Blackwell%20vRAM%20and%20other%20technical%20specs https://xplained.vercel.app/ask?q=Nvidia%20Blackwell%20vRAM%...

24 points by mfiguiere 3y ago

The article makes no mention at all of how much VRAM they might have. Other articles state plans for up to 192GB of HMB3e, and a power draw of 1000 watts.

deleted 3y ago
[deleted]

klysm 3y ago

It's terrifying to think about how many watts of power are being spent on generating useless trash

epistasis 3y ago
This is a bit vague, are you talking about this new GPU/LLMs, social media, industrial society, or something else?
vasco 3y ago
Most human consumption is useless trash by some metric.
reaperman 3y ago
Fun fact: each human uses an average 100W of power and generates mostly useless crap.
- visarga 3y ago
  > each human uses an average 100W of power Does that include heating, electricity and fuel spent for the benefit of the human? How about infrastructure? Externalities are huge.

deleted 3y ago

[deleted]

epistasis 3y ago

> next-gen NVLink switch that lets 576 GPUs talk to each other, with 1.8 terabytes per second of bidirectional bandwidth. Well that's a pretty big step up from the 18 for the H100...

sp332 3y ago

I know it’s only FP4, but 1.4 exaflops in one rack is still crazy.

alecco 3y ago
With scaling and some tricks, FP4 inference can be very close to 16 bit. And most software out there is still missing on FP8 lol
- cyanydeez 3y ago
  Doesn't that suggest the training method is flawed?
  - sp332 3y ago
    For training, you need to follow a gradient, and usually the gradient is small enough that you need the precision. Training in 8-bit is fairly new but Google and now Grok seem to have gotten good results.
    - alecco 3y ago
      And Microsoft.
    - cyanydeez 3y ago
      So these cards are geared towards inference ? There's a register article showing amds new instinct accelerators have much better.FP64
      - sp332 3y ago
        The article has numbers for both training, I think in FP8, and inference in FP4.
- deleted 3y ago
  [deleted]

jpeggtulsa 3y ago

[dead]

_yo2u 3y ago
you mean the chip that only has 44GB of memory on one chip and doesn't mention how fast chips can talk between each other?
- jpeggtulsa 3y ago
  Why cut up a a wafer of chips, package each with HBM, put the package on a board, connect to CPUs with a fabric, then tie them all back together with networking chips and cables? Their 3 new clusters are the top 3 biggest AI training platforms in the world. Comparing the WSE-3 against Nvidia's H100. "It's got 52 times more cores. It's got 800 times more memory on chip. It's got 7,000 times more memory bandwidth and more than 3,700 times more fabric bandwidth. But they don't sell the chips, they build the clusters and sell the compute power. Except for a couple they built in Dubai. Plus, newer models are moving from Transformer to Mamba and don't need as much memory, because they save the important information, not everything it's been trained on.

shouche 3y ago

Explained by AI https://xplained.vercel.app/ask?q=Nvidia%20Blackwell https://xplained.vercel.app/ask?q=Nvidia%20Blackwell Additional details on vRAM and spec: https://xplained.vercel.app/ask?q=Nvidia%20Blackwell%20vRAM%20and%20other%20technical%20specs https://xplained.vercel.app/ask?q=Nvidia%20Blackwell%20vRAM%...