6 ms·

Note that in terms of pure compute performance the new 16nm Nvidia GTX 1080 (2560 shaders at up to 1733 MHz = 8873 SP GFLOPS) barely equals the performance of t

by mrb 11y ago

Note that in terms of pure compute performance the new 16nm Nvidia GTX 1080 (2560 shaders at up to 1733 MHz = 8873 SP GFLOPS) barely equals the performance of the previous-generation 28nm AMD Fury X (4096 shaders at up to 1050 MHz = 8602 SP GFLOPS). Of course the 16nm chip does so at a significantly lower TDP (180 Watt) than the 28nm chip (275 Watt), so it will be interesting to see what Nvidia can achieve at a higher thermal/power envelope with a more high-end card... I am waiting impatiently to see how AMD's upcoming 14nm/16nm Polaris chips will fare, but from the looks of it it seems like Polaris will beat Nvidia in terms of GFLOPS per Watt.

redtuesday 11y ago
Yeah, it will be interesting to see how good AMD's Polaris chips will be, especially when it comes to energy efficiency. Fury was a step in the right direction (I'm wondering how much of this can be attributed to HBM's lower voltage etc.). But I'm even more interested in AMD Vega - the high end competitor to Nvidias GP 100 that is equipped with HBM2. Especially if AMD has an advantage over Nvidia when it comes to implementing HBM, since they developed it together with SK Hynx and have previous experience with Fury, or if this doesn't make a difference. Vega is also exiting because it should be released when AMD's Zen CPU's will be released, and there were rumors about a HPC Zen APU with Vega and HBM.
- yohui 11y ago
  Both Vega and Zen are supposed to be due sometime next year, correct?
  - redtuesday 11y ago
    Yes, Zen for the desktop (Summit Ridge) is even listed for end of 2016 [1], followed by first Zen APU's (Raven Ridge) sometime 2017 and Vega early 2017. [2] [1] http://imgur.com/rRjKZDk http://imgur.com/rRjKZDk [2] http://i.imgur.com/lkjgayO.jpg?1 http://i.imgur.com/lkjgayO.jpg?1
- mtgx 11y ago
  If we're discussing future GPUs, does anyone know what exactly AMD means by "next-gen memory" when it talks about Navi? At first I thought it means that HBM2 would be available across their whole lineup, but I think if that was the case they would've just said so. I've also read a bit about how Nvidia wanted to go with HMC (Hyper Memory Cube) instead of HBM, but it seems it's still at least twice as expensive/GB, and they needed to go with high-end GPUs that have at least 16GB. HMC is not even at 8GB yet. Intel also seems to have adopted HMC for some servers. So is the "next-gen" memory HMC, or something else? AMD is supposed to come out with it in 2018, hopefully at 10nm.
  - redtuesday 11y ago
    Was the difference between HMC and HBM not basically HMC had good latency comparable to normal main RAM like DDR3 plus good bandwith (up to 400 GB/s), while HBM had worse latency than DDR3 (like GDDR5) with a large bus and really good bandwith (1 TB/s)? Will indeed be interesting to see what AMD means with next gen memory.
    - Tuna-Fish 11y ago
      No. HMC is fully buffered and provides a packet-based interface to the CPU it's connected into. This means that the CPU doesn't need to know anything about the memory it uses, and saves die space on the CPU side, but it adds latency. HBM has lower latency than HMC. Also, GDDR5 does not have higher latency than DDR3. The memory arrays themselves are the same and exactly as fast, and the bus doesn't add any extra latency. GDDR5 doesn't trade latency for bandwidth, it trades density for bandwidth. Things built to use GDDDR5 often have much higher latencies than things built to use DDR3, but this has nothing to do with the memory and instead about how the memory controllers in GPUs delay accesses to merge ones close to each other to optimize for bandwidth.
      - redtuesday 11y ago
        Thank you for the detailed explanation. Always thought the added latency came from GDDR5. Good to know.
sp332 11y ago
The Fury X is AMD's current flagship, not last-gen.
- mrb 11y ago
  I meant previous-gen chip fab process (28nm) vs. new-gen (14/16nm).
tcoppi 11y ago
AMD chips have almost always had a lead over Nvidia from a per-flops perspective, and AMD's architecture is definitely ahead of Nvidia as far as supporting integer operations and bitwise ops efficiently(see their dominance of cryptocurrency mining). However, Nvidia's software tooling has always been superb, and they basically own the GPGPU-for-HPC space because of it. On top of that, one of the biggest growth areas of consumers of GPGPU in recent years has been "deep learning", for which Nvidia has embraced and has great tooling and software for their hardware. AMD has potential to compete with Nvidia in this area, but they don't seem to have the resources to develop optimized Neural Network software to run on their GPUs like Nvidia does, so they are way behind. Basically, GFLOPS per Watt is nice and all, but if nothing you care about runs on the hardware, you won't buy it. AMD seems to have been stuck in this position for HPC, and it doesn't seem to be changing sadly.
- valarauca1 11y ago
  AMD pulls off better compute in pure GFLOPS but you'll have to extract that compute power in OpenCL. AMD still doesn't have a good answer to CUDA.
  - Athas 11y ago
    OpenCL is however a fairly nice target for compilers, which opens up different ways of using it. Witness also PyOpenCL versus PyCUDA - both are essentially equally convenient to use, while raw OpenCL C is a nightmare compared to CUDA C.
    - m_mueller 11y ago
      I think the near future in compilers is going to be dominated by accelerator capable LLVM variants (such as NVVM). Ideally, all Nvidia, AMD and Intel would and should need to do is to provide LLLVM based backends. Frontends just need a sane way of specifying SIMT. And no, loop + directive based stuff like OpenMP and OpenACC is IMO just a stopgap measure in order to serve higher ups the lie that portation is going to be swift and easy. There's basically one sane way to deal with data parallel problems, and that's to treat what's going on in one core as a scalar program with precomputed indices (i.e. the CUDA / OpenCL way). Python has some good support for this already. I haven't done a project like this yet, but if setting up something fresh I'd immediately jump on NumPy interacting with my own kernels and some Python glue for the node level parallelism.
varelse 11y ago
The most interesting thing about AMD is their ability to create GPUs with superior synthetic specs than their NVIDIA competitor and yet they lose across the board in anything but embarrassingly parallel applications. Not only do they lose, but they lose big. Here are two examples. First, the engine behind Folding@Home: https://simtk.org/plugins/moinmoin/openmm/BenchmarkOpenMMDHFR https://simtk.org/plugins/moinmoin/openmm/BenchmarkOpenMMDHF... See also AlexNet (Deep Learning's most prominent benchmark): https://github.com/amd/OpenCL-caffe https://github.com/amd/OpenCL-caffe versus GTX TitanX: https://github.com/soumith/convnet-benchmarks https://github.com/soumith/convnet-benchmarks TLDR: AMD losing by a factor of 2 or more... So unless this changes dramatically, NVIDIA IMO will continue to dominate. But hey, they both crush Xeon Phi and FPGAs so it doesn't suck to be #2 when #3 is Intel.
- listic 11y ago
  Which FPGAs in which workloads do they crush?
  - varelse 11y ago
    Arria 10: http://www.nextplatform.com/2015/08/27/microsoft-extends-fpga-reach-from-bing-to-deep-learning/ http://www.nextplatform.com/2015/08/27/microsoft-extends-fpg... 600 images/s inference (highest ever publically reported), projected to be as high as 900 images/s. ~$5000 per FPGA. Compare and contrast with TitanX/GTX 980 TI now topping 5000 images/s. GTX 1080 will only be faster than this and $600. https://github.com/soumith/convnet-benchmarks https://github.com/soumith/convnet-benchmarks And so far, no FPGA training numbers. But here are some distributed training numbers for $10Kish Xeon servers: https://software.intel.com/en-us/articles/caffe-training-on-multi-node-distributed-memory-systems-based-on-intel-xeon-processor-e5 https://software.intel.com/en-us/articles/caffe-training-on-... 64 of them train Alexnet in ~5 hours. A DGX-1 does it in 2. Don't have/Can't get a $129K DGX-1? Fine, buy a Big Sur from one of several vendors and throw in 8 GTX 1080s as soon as they're out, implement "One Weird Trick", and you'll do it in <3. That ought to run you about $30-$35K versus >$600K for those 64 Xeon servers. FPGAs OTOH are fantastic at low memory bandwidth embarrassingly parallel computation like bitcoin mining. Deep Learning is not such a domain.

by mrb 11y ago

redtuesday 11y ago

Yeah, it will be interesting to see how good AMD's Polaris chips will be, especially when it comes to energy efficiency. Fury was a step in the right direction (I'm wondering how much of this can be attributed to HBM's lower voltage etc.). But I'm even more interested in AMD Vega - the high end competitor to Nvidias GP 100 that is equipped with HBM2. Especially if AMD has an advantage over Nvidia when it comes to implementing HBM, since they developed it together with SK Hynx and have previous experience with Fury, or if this doesn't make a difference. Vega is also exiting because it should be released when AMD's Zen CPU's will be released, and there were rumors about a HPC Zen APU with Vega and HBM.

yohui 11y ago
Both Vega and Zen are supposed to be due sometime next year, correct?
- redtuesday 11y ago
  Yes, Zen for the desktop (Summit Ridge) is even listed for end of 2016 [1], followed by first Zen APU's (Raven Ridge) sometime 2017 and Vega early 2017. [2] [1] http://imgur.com/rRjKZDk http://imgur.com/rRjKZDk [2] http://i.imgur.com/lkjgayO.jpg?1 http://i.imgur.com/lkjgayO.jpg?1
mtgx 11y ago
If we're discussing future GPUs, does anyone know what exactly AMD means by "next-gen memory" when it talks about Navi? At first I thought it means that HBM2 would be available across their whole lineup, but I think if that was the case they would've just said so. I've also read a bit about how Nvidia wanted to go with HMC (Hyper Memory Cube) instead of HBM, but it seems it's still at least twice as expensive/GB, and they needed to go with high-end GPUs that have at least 16GB. HMC is not even at 8GB yet. Intel also seems to have adopted HMC for some servers. So is the "next-gen" memory HMC, or something else? AMD is supposed to come out with it in 2018, hopefully at 10nm.
- redtuesday 11y ago
  Was the difference between HMC and HBM not basically HMC had good latency comparable to normal main RAM like DDR3 plus good bandwith (up to 400 GB/s), while HBM had worse latency than DDR3 (like GDDR5) with a large bus and really good bandwith (1 TB/s)? Will indeed be interesting to see what AMD means with next gen memory.
  - Tuna-Fish 11y ago
    No. HMC is fully buffered and provides a packet-based interface to the CPU it's connected into. This means that the CPU doesn't need to know anything about the memory it uses, and saves die space on the CPU side, but it adds latency. HBM has lower latency than HMC. Also, GDDR5 does not have higher latency than DDR3. The memory arrays themselves are the same and exactly as fast, and the bus doesn't add any extra latency. GDDR5 doesn't trade latency for bandwidth, it trades density for bandwidth. Things built to use GDDDR5 often have much higher latencies than things built to use DDR3, but this has nothing to do with the memory and instead about how the memory controllers in GPUs delay accesses to merge ones close to each other to optimize for bandwidth.
    - redtuesday 11y ago
      Thank you for the detailed explanation. Always thought the added latency came from GDDR5. Good to know.

sp332 11y ago

The Fury X is AMD's current flagship, not last-gen.

mrb 11y ago
I meant previous-gen chip fab process (28nm) vs. new-gen (14/16nm).

tcoppi 11y ago

AMD chips have almost always had a lead over Nvidia from a per-flops perspective, and AMD's architecture is definitely ahead of Nvidia as far as supporting integer operations and bitwise ops efficiently(see their dominance of cryptocurrency mining). However, Nvidia's software tooling has always been superb, and they basically own the GPGPU-for-HPC space because of it. On top of that, one of the biggest growth areas of consumers of GPGPU in recent years has been "deep learning", for which Nvidia has embraced and has great tooling and software for their hardware. AMD has potential to compete with Nvidia in this area, but they don't seem to have the resources to develop optimized Neural Network software to run on their GPUs like Nvidia does, so they are way behind. Basically, GFLOPS per Watt is nice and all, but if nothing you care about runs on the hardware, you won't buy it. AMD seems to have been stuck in this position for HPC, and it doesn't seem to be changing sadly.

valarauca1 11y ago
AMD pulls off better compute in pure GFLOPS but you'll have to extract that compute power in OpenCL. AMD still doesn't have a good answer to CUDA.
- Athas 11y ago
  OpenCL is however a fairly nice target for compilers, which opens up different ways of using it. Witness also PyOpenCL versus PyCUDA - both are essentially equally convenient to use, while raw OpenCL C is a nightmare compared to CUDA C.
  - m_mueller 11y ago
    I think the near future in compilers is going to be dominated by accelerator capable LLVM variants (such as NVVM). Ideally, all Nvidia, AMD and Intel would and should need to do is to provide LLLVM based backends. Frontends just need a sane way of specifying SIMT. And no, loop + directive based stuff like OpenMP and OpenACC is IMO just a stopgap measure in order to serve higher ups the lie that portation is going to be swift and easy. There's basically one sane way to deal with data parallel problems, and that's to treat what's going on in one core as a scalar program with precomputed indices (i.e. the CUDA / OpenCL way). Python has some good support for this already. I haven't done a project like this yet, but if setting up something fresh I'd immediately jump on NumPy interacting with my own kernels and some Python glue for the node level parallelism.

varelse 11y ago

The most interesting thing about AMD is their ability to create GPUs with superior synthetic specs than their NVIDIA competitor and yet they lose across the board in anything but embarrassingly parallel applications. Not only do they lose, but they lose big. Here are two examples. First, the engine behind Folding@Home: https://simtk.org/plugins/moinmoin/openmm/BenchmarkOpenMMDHFR https://simtk.org/plugins/moinmoin/openmm/BenchmarkOpenMMDHF... See also AlexNet (Deep Learning's most prominent benchmark): https://github.com/amd/OpenCL-caffe https://github.com/amd/OpenCL-caffe versus GTX TitanX: https://github.com/soumith/convnet-benchmarks https://github.com/soumith/convnet-benchmarks TLDR: AMD losing by a factor of 2 or more... So unless this changes dramatically, NVIDIA IMO will continue to dominate. But hey, they both crush Xeon Phi and FPGAs so it doesn't suck to be #2 when #3 is Intel.

listic 11y ago
Which FPGAs in which workloads do they crush?
- varelse 11y ago
  Arria 10: http://www.nextplatform.com/2015/08/27/microsoft-extends-fpga-reach-from-bing-to-deep-learning/ http://www.nextplatform.com/2015/08/27/microsoft-extends-fpg... 600 images/s inference (highest ever publically reported), projected to be as high as 900 images/s. ~$5000 per FPGA. Compare and contrast with TitanX/GTX 980 TI now topping 5000 images/s. GTX 1080 will only be faster than this and $600. https://github.com/soumith/convnet-benchmarks https://github.com/soumith/convnet-benchmarks And so far, no FPGA training numbers. But here are some distributed training numbers for $10Kish Xeon servers: https://software.intel.com/en-us/articles/caffe-training-on-multi-node-distributed-memory-systems-based-on-intel-xeon-processor-e5 https://software.intel.com/en-us/articles/caffe-training-on-... 64 of them train Alexnet in ~5 hours. A DGX-1 does it in 2. Don't have/Can't get a $129K DGX-1? Fine, buy a Big Sur from one of several vendors and throw in 8 GTX 1080s as soon as they're out, implement "One Weird Trick", and you'll do it in <3. That ought to run you about $30-$35K versus >$600K for those 64 Xeon servers. FPGAs OTOH are fantastic at low memory bandwidth embarrassingly parallel computation like bitcoin mining. Deep Learning is not such a domain.