5 ms·

This seems fast... TF32 ....... 1,000 TFLOPS (tensor core) FP64/FP32 ... 60 TFLOPS I am more interested in the 144-core Grace CPU Superchip. nV

This seems fast... TF32 ....... 1,000 TFLOPS (tensor core) FP64/FP32 ... 60 TFLOPS I am more interested in the 144-core Grace CPU Superchip. nVidia is getting into the CPU business...

cjbgkagh 5y ago
I think the 1PFLOPS figure for TF32 is with sparsity, which should be called out in the name. Maybe ‘TFS32’? I mainly use dense FP16 so the 1PFLOPS for that looks pretty good.
- lostmsu 5y ago
  Asked elsewhere, but why FP16 as opposed to BF16?
  - cjbgkagh 5y ago
    I'm using older Turing GPUs BF16 would require Ampere. The weights in my models tend to be normalized so the fraction would be more important than the exponent so I would probably still use FP16. I would need to test it though.
Melatonic 5y ago
Same - plus its SUPER
deleted 5y ago
[deleted]
macrolocal 5y ago
50% sparsity and rated at 700W. The new DGX is 10kW!
- wmwmwm 5y ago
  I was recently researching how you'd host systems like this in a datacentre and was blown away to find out that you can cool 40kW in a single air cooled rack - this might be old news for many, but it was 2x or 3x what I expected! Glad I'm not paying the electricity bill :)
  - jmole 5y ago
    Here's what a propane heater of similar output looks like: https://www.amazon.com/Dura-Heat-Propane-Forced-Heater/dp/B00427WQ36 https://www.amazon.com/Dura-Heat-Propane-Forced-Heater/dp/B0...
    - HelloNurse 5y ago
      Most of the propane heater is a fan in a tube, the flame is probably quite smaller than a CPU package.
      - baq 5y ago
        I've got an 8kW wood stove and that thing gets rather hot to touch - as in, you will get a blister... 40kW is a small city car worth of power.
        lIIIllllIIII 5y ago
        If you think about it, cars can manage cooling 200+kW with a radiator - and a lot of airflow. That's still an amazing amount of power though. I can't help thinking about the kind of sear you could get on a steak with 40kW of power :D
  - namibj 5y ago
    Supermicro will sell you 40 kW CPU TDP in a single air-cooled rack, not counting the rest of the servers.
  - sliken 5y ago
    Possible, yes. Easy, no. Don't assume any ole air cooled rack (or datacenter) can manage 40k per rack. It makes it really important to do the hot isle/cold isle well, manage the airflow carefully, block any empty rack units, etc. etc. etc.

This seems fast... TF32 ....... 1,000 TFLOPS (tensor core) FP64/FP32 ... 60 TFLOPS I am more interested in the 144-core Grace CPU Superchip. nVidia is getting into the CPU business...

cjbgkagh 5y ago

I think the 1PFLOPS figure for TF32 is with sparsity, which should be called out in the name. Maybe ‘TFS32’? I mainly use dense FP16 so the 1PFLOPS for that looks pretty good.

lostmsu 5y ago
Asked elsewhere, but why FP16 as opposed to BF16?
- cjbgkagh 5y ago
  I'm using older Turing GPUs BF16 would require Ampere. The weights in my models tend to be normalized so the fraction would be more important than the exponent so I would probably still use FP16. I would need to test it though.

Melatonic 5y ago

Same - plus its SUPER

deleted 5y ago

[deleted]

macrolocal 5y ago

50% sparsity and rated at 700W. The new DGX is 10kW!

wmwmwm 5y ago
I was recently researching how you'd host systems like this in a datacentre and was blown away to find out that you can cool 40kW in a single air cooled rack - this might be old news for many, but it was 2x or 3x what I expected! Glad I'm not paying the electricity bill :)
- jmole 5y ago
  Here's what a propane heater of similar output looks like: https://www.amazon.com/Dura-Heat-Propane-Forced-Heater/dp/B00427WQ36 https://www.amazon.com/Dura-Heat-Propane-Forced-Heater/dp/B0...
  - HelloNurse 5y ago
    Most of the propane heater is a fan in a tube, the flame is probably quite smaller than a CPU package.
    - baq 5y ago
      I've got an 8kW wood stove and that thing gets rather hot to touch - as in, you will get a blister... 40kW is a small city car worth of power.
      - lIIIllllIIII 5y ago
        If you think about it, cars can manage cooling 200+kW with a radiator - and a lot of airflow. That's still an amazing amount of power though. I can't help thinking about the kind of sear you could get on a steak with 40kW of power :D
- namibj 5y ago
  Supermicro will sell you 40 kW CPU TDP in a single air-cooled rack, not counting the rest of the servers.
- sliken 5y ago
  Possible, yes. Easy, no. Don't assume any ole air cooled rack (or datacenter) can manage 40k per rack. It makes it really important to do the hot isle/cold isle well, manage the airflow carefully, block any empty rack units, etc. etc. etc.