4 ms·

The DGX-1 is available for a cool $129k: http://www.nvidia.com/object/deep-learning-system.html http://www.nvidia.com/object/deep-learning-system.html Correct

The DGX-1 is available for a cool $129k: http://www.nvidia.com/object/deep-learning-system.html http://www.nvidia.com/object/deep-learning-system.html Correct me if I'm wrong, but I think it's basically a couple hundred NVIDIA 10-series cards strapped together with a full custom NVIDIA software stack.

ansible 10y ago
No, I don't think that's a good characterization. Sure, there's been improvements to the computational performance. But the big deal (to me at least) is the unified memory model between the GPUs and the Xeon host processors. This makes a lot of things easier to code for on a single system, and it makes multi-system applications easier to scale. This is because you're streaming data in over the network (10G Ethernet) and then the GPUs can operate on it without an extra copy step. The copy step also implies more management and shuffling around of the data you're operating on.
tlb 10y ago
A DGX-1 box has 8 Pascal GPUs. The reason it costs a lot more than 8 GTX 1080s is the remarkable interconnect and memory bandwidth.
madenine 10y ago
Pretty much. It uses their P100 HPC cards instead of consumer grade cards (8x P100s), plus two Xeon E5v4 chips, half a TB of RAM and 7.5 TBs of SSD storage - all wrapped up nicely configured for you with their CPU-GPU speed up stack. I believe the only way to get P100s right now is in the DGX-1, so there's that.
psb217 10y ago
The P100s have full support for half-precision (i.e. 16 bit) floating point ops. This can mean ~2x improvements in speed and memory usage in comparison to the Pascal TitanX, which is the top "consumer" card. This difference is significant for almost any machine learning workload, which is what a lot of these cards will be used for. NVIDIA gimped half-precision on the consumer cards to drive datacenters, hedge funds, machine learning companies, etc. towards the "professional" cards (and their huge markup).
- p1esk 10y ago
  FP16 performance is only relevant until people figure out how to train NNs using INT8. See, for example, [1] for recent advances in that direction. After that, it's going to be mostly about memory size and bandwidth. [1] https://arxiv.org/abs/1603.01025 https://arxiv.org/abs/1603.01025
- yahma 10y ago
  First NVIDIA solidified their Monopoly by forcing CUDA... then they gimped half-precision on consumer cards. We really need some more Frameworks that work with OpenCL, so that we can have some competition from AMD, who's consumer cards are not gimped.
  - Tom1971 10y ago
    Gimping, in this case, is actually: adding hardware, that costs quite a bit of silicon area, on one chip that will probably never be sold as a consumer GPU. I don't see the issue with a company making a very high-end product, adding stuff that doesn't have good use for consumers, and asking extra money for their effort. AMD doesn't have double speed FP16 on its current FPUs either. The latest version has FP16 at the same speed as FP32, but if you're doing that you might as well use FP32 always. And let's not forget: the Nvidia consumer GPU have deep learning quad int8 operations enabled at all time. They didn't need to do that and could have reserved it for their Tesla product line only.

ansible 10y ago

No, I don't think that's a good characterization. Sure, there's been improvements to the computational performance. But the big deal (to me at least) is the unified memory model between the GPUs and the Xeon host processors. This makes a lot of things easier to code for on a single system, and it makes multi-system applications easier to scale. This is because you're streaming data in over the network (10G Ethernet) and then the GPUs can operate on it without an extra copy step. The copy step also implies more management and shuffling around of the data you're operating on.

tlb 10y ago

A DGX-1 box has 8 Pascal GPUs. The reason it costs a lot more than 8 GTX 1080s is the remarkable interconnect and memory bandwidth.

madenine 10y ago

Pretty much. It uses their P100 HPC cards instead of consumer grade cards (8x P100s), plus two Xeon E5v4 chips, half a TB of RAM and 7.5 TBs of SSD storage - all wrapped up nicely configured for you with their CPU-GPU speed up stack. I believe the only way to get P100s right now is in the DGX-1, so there's that.

psb217 10y ago

The P100s have full support for half-precision (i.e. 16 bit) floating point ops. This can mean ~2x improvements in speed and memory usage in comparison to the Pascal TitanX, which is the top "consumer" card. This difference is significant for almost any machine learning workload, which is what a lot of these cards will be used for. NVIDIA gimped half-precision on the consumer cards to drive datacenters, hedge funds, machine learning companies, etc. towards the "professional" cards (and their huge markup).

p1esk 10y ago
FP16 performance is only relevant until people figure out how to train NNs using INT8. See, for example, [1] for recent advances in that direction. After that, it's going to be mostly about memory size and bandwidth. [1] https://arxiv.org/abs/1603.01025 https://arxiv.org/abs/1603.01025
yahma 10y ago
First NVIDIA solidified their Monopoly by forcing CUDA... then they gimped half-precision on consumer cards. We really need some more Frameworks that work with OpenCL, so that we can have some competition from AMD, who's consumer cards are not gimped.
- Tom1971 10y ago
  Gimping, in this case, is actually: adding hardware, that costs quite a bit of silicon area, on one chip that will probably never be sold as a consumer GPU. I don't see the issue with a company making a very high-end product, adding stuff that doesn't have good use for consumers, and asking extra money for their effort. AMD doesn't have double speed FP16 on its current FPUs either. The latest version has FP16 at the same speed as FP32, but if you're doing that you might as well use FP32 always. And let's not forget: the Nvidia consumer GPU have deep learning quad int8 operations enabled at all time. They didn't need to do that and could have reserved it for their Tesla product line only.