4 ms·

If the NVIDIA DGX A100 won't restore your faith in cool computers then nothing will. https://www.youtube.com/watch?v=TJcKYUTaBtg https://www.youtube.com/watch?v

If the NVIDIA DGX A100 won't restore your faith in cool computers then nothing will. https://www.youtube.com/watch?v=TJcKYUTaBtg https://www.youtube.com/watch?v=TJcKYUTaBtg

ajfjrbfbf 6y ago
Can this run GPT3? Also a 6.5kW computer must need quite some cooling system...
- fennecfoxen 6y ago
  Let's see. According to https://bdtechtalks.com/2020/09/21/gpt-3-economy-business-model/ https://bdtechtalks.com/2020/09/21/gpt-3-economy-business-mo... GPT-3 was run on V100s, which are the previous generation server GPUs ("Volta" vs "Ampere"). "GPT-3 uses half-precision floating-point variables at 16 bits per parameter... At around $130,000, DGX-1 is short on VRAM (8×16 GB), but has all the other components for a solid performance on GPT-3." (DGX-1 is the previous chassis NVIDIA offered for hosting V100s.) So I tentatively think the answer is yes, a single DGX-A100 probably can run GPT-3 comfortably. You'll still want a cluster of them to train it, though, because they note training takes about 355 V100-years.
  - p1esk 6y ago
    You don’t need any GPUs to run gpt-3. You can run it on a regular server with >512GB RAM.
    - fennecfoxen 6y ago
      I can only comment that this is at odds with the information at the link I supplied, and I cannot tell you for certain whether that information is actually accurate.
      - qayxc 6y ago
        The link only mentions requirements to run inference at "decent speeds", without going into details about what they consider to be decent speeds. In principle you can of course run any model on any hardware that has enough RAM. Whether the inference performance is acceptable depends your particular application. I'd argue that for most non-interactive use cases, inference speed doesn't really matter and the cost benefit from running on CPUs vs GPUs might be worth it.
wolfgke 6y ago
This will probably not be a very open system. In this sense: no, it won't restore my faith.
- qayxc 6y ago
  Since when were SGI or HP workstations open systems? These things were the very definition of walled garden.
  - wolfgke 6y ago
    In my youth, I was inspired by different systems than very closed workstations.

If the NVIDIA DGX A100 won't restore your faith in cool computers then nothing will. https://www.youtube.com/watch?v=TJcKYUTaBtg https://www.youtube.com/watch?v=TJcKYUTaBtg

ajfjrbfbf 6y ago

Can this run GPT3? Also a 6.5kW computer must need quite some cooling system...

fennecfoxen 6y ago
Let's see. According to https://bdtechtalks.com/2020/09/21/gpt-3-economy-business-model/ https://bdtechtalks.com/2020/09/21/gpt-3-economy-business-mo... GPT-3 was run on V100s, which are the previous generation server GPUs ("Volta" vs "Ampere"). "GPT-3 uses half-precision floating-point variables at 16 bits per parameter... At around $130,000, DGX-1 is short on VRAM (8×16 GB), but has all the other components for a solid performance on GPT-3." (DGX-1 is the previous chassis NVIDIA offered for hosting V100s.) So I tentatively think the answer is yes, a single DGX-A100 probably can run GPT-3 comfortably. You'll still want a cluster of them to train it, though, because they note training takes about 355 V100-years.
- p1esk 6y ago
  You don’t need any GPUs to run gpt-3. You can run it on a regular server with >512GB RAM.
  - fennecfoxen 6y ago
    I can only comment that this is at odds with the information at the link I supplied, and I cannot tell you for certain whether that information is actually accurate.
    - qayxc 6y ago
      The link only mentions requirements to run inference at "decent speeds", without going into details about what they consider to be decent speeds. In principle you can of course run any model on any hardware that has enough RAM. Whether the inference performance is acceptable depends your particular application. I'd argue that for most non-interactive use cases, inference speed doesn't really matter and the cost benefit from running on CPUs vs GPUs might be worth it.

wolfgke 6y ago

This will probably not be a very open system. In this sense: no, it won't restore my faith.

qayxc 6y ago
Since when were SGI or HP workstations open systems? These things were the very definition of walled garden.
- wolfgke 6y ago
  In my youth, I was inspired by different systems than very closed workstations.