7 ms·

The Expanding Ecosystem For GPU Computing

58 points by Katydid 11y ago

v4n4d1s 11y ago
We use CST microwave studio (https://www.cst.com/Products/CSTMWS https://www.cst.com/Products/CSTMWS) for a project. Calculation time on a Xeon E5-2680 was about 50-60 hours per simulation, together with a Tesla K80 it was reduced to about 4-5 hours. Pretty impressive. Adoption rate isn't very high, since most applications still don't use that power.
fmeyer 11y ago
NVidia gave me a Titan X for my learning experiments, and after all these years, I'm really excited about finish building a PC again.
- jawilson2 11y ago
  NVidia has been great for this. Back in 2009 they gave me 2 Teslas (C2060) and 2 Quadro FX 5800 cards. Then, I was able to get some results from those to get an NVidia Professor Partnership grant of $25000 (as a post doc). They had a large part in launching my engineering career doing HPC.
danieldk 11y ago
nVidia's Tesla processors and high-end GPUs are really stellar. We recently bought a machine with a Tesla K20c with some colleagues and the speed-up for neural net training has been tremendous. E.g. for some of my nets (NLP tasks) training time has gone from hours to 15-30 minutes. This is great when you are just experimenting with different approaches to a problem and want to iterate quickly. It's also great that nVidia hasn't been sitting on their laurels and released a library (cuDNN) of NN primitives that nearly everyone has been rebasing their packages on.
ihsw 11y ago
Why does NVidia dominate GPU computing? Where is AMD in this space? I was under the impression that AMD's strongest R&D in the past few years has been directed at hybrid consumer desktops, surely this should translate directly into their server processing products.
- wyldfire 11y ago
  IMO it's all due to the developer ecosystem. NVIDIA had a big head start with CUDA. AMD effectively embraced CUDA and transformed the best of CUDA into OpenCL (jointly with NVIDIA, Apple, via Khronos). NVIDIA's OpenCL support lags the standard, while they keep improving CUDA. NVIDIA probably feels commitment to their early adopters and doesn't want to divide up their engineering capability. Meanwhile NVIDIA also seems to have really good marketing behind their GPU software stack. Lots of people who use CUDA have no idea that OpenCL exists and no idea that porting between CUDA and OCL is not super difficult. (It's been years since I've looked at CUDA it might have diverged more significantly since). AMD's stability (on linux at least) is pretty poor. I don't know if it hurts adoption much, but it can't help. Also, AMD absolutely dominates on integer workloads which is why it had been used so much for cryptocoin hashing machines prior to the widespread availability of ASICs for most algorithms. On floating point workloads, NVIDIA might do better than AMD, and those seem much more common in science than integer math.
  - nhaehnle 11y ago
    > AMD's stability (on linux at least) is pretty poor. Have you had any concrete issues recently? This may be one of these things where public perception changes slowly, unfortunately.
    - wyldfire 11y ago
      Yes, I have. :( I believe that they may be improving, but in general I have seen far fewer functionality problems from NVIDIA than AMD. Generation after generation, it seems to replay itself.
      - nhaehnle 11y ago
        Can you give details? If it's with the open driver stack, I may be able to help. Contact me at username at gmail if you like.
    - bainsfather 11y ago
sorenjan 11y ago
AMD isn't even mentioned in this article, and I recently read that nobody uses AMD for machine learning. Is this only down to the proprietary Cuda having more momentum than OpenCL? I've used both, albeit not very much, and my impression was that Cuda was a bit easier to write but OpenCL didn't have any deal breakers. I know AMD cards were the most popular for crypto currencies, so it's not like they're always the slowest. Granted, I only have experience with each company's consumer cards, and I think Nvidia holds their GeForce cards back more than AMD does their Radeon cards concerning compute performance. Why is Nvidia this dominant, and why are so many researchers using a proprietary API like Cuda when there's open alternatives? Related to this, are anybody using Intel's Xeon Phi? Tianhe-2 [0], the worlds fastest supercomputer since June 2013 use them as co-processors, but I rarely if ever hear about them in other projects. [0] https://en.wikipedia.org/wiki/Tianhe-2 https://en.wikipedia.org/wiki/Tianhe-2
- DHJSH 11y ago
  We find the price performance of NVIDIA hard to beat, combined with excellent compiler and developer support. We get better performance writing CUDA code directly than OpenCL, and it's easier for us to develop and debug. Yes, it's "proprietary" but we need to get our job done.
  - sorenjan 11y ago
    It's in Nvidia's best interest to have their own Cuda be faster than OpenCL. Question is, which is faster: OpenCL on AMD, or Cuda on Nvidia?
- lordCarbonFiber 11y ago
  I have a little experience in GPU programming so I can address your first question. ;TLDR It is mostly due to momentum. The difference in momentum stems from a difference in design paradigms. OpenCL is just a library that you can link to can compile using GCC. Because the code you're writing is C++ syntactically you're often jumping through a lot more hoops. Nvidia uses their own compiler, so when you're writing cuda programs you have something that looks a lot like C++, however they have the freedom to extend the syntax to ease development. The marketing arm of Nvidia, combined with the ease of use helped get Cuda the defacto research platform. Once that ball was rolling, it becomes easier to find cuda teachers and cuda examples so it's naturally become more prolific.
marssaxman 11y ago
I've been ignoring GPUs for years, waiting for open standards, public documentation, and open-source drivers. I'm really disappointed that so much work is still being poured into these mysterious proprietary architectures. We wouldn't tolerate this anywhere else; why are GPUs allowed to be an exception?
- jjaredsimpson 11y ago
  Because ideology doesn't get work done. This is an example of making the perfect the enemy of the good. I assume you've been ignoring GPUs for years because you don't work in a problem space where you can leverage their power. Lots of other people however are in those spaces getting work done.
  - marssaxman 11y ago
    I actually used to do a fair bit of work involving embarrassingly parallel algorithms on floating-point numbers - audio & video synthesis stuff. My last project was a little language/compiler system built around vectorization. I stopped working on these problems after it became clear that GPUs were now the best way to accomplish them. I thought I'd wait a year or two and get back into it after the relevant documentation came out. It's been over a decade and I'm still waiting. Fortunately, there are plenty of other interesting problems to work on. Ideology may not get work done, but secrecy also prevents work from getting done. I can't continue with the line of work I was pursuing back then as long as the relevant interfaces are wrapped up in a binary blob.
PascLeRasc 11y ago
It seems to me that there would be a massive demand to be able to work entirely in OS X by researchers for whom computing is a secondary task (I've seen this a lot in my neurobiology department). Most of these people are only knee-deep in the computing world, and learning Linux is not worth it to them. I know several researchers just getting into machine learning for their image processing/CNN projects, and they're just living with the speed of a 3+ year old Mac Pro, though they have the funding and desire to add plenty of GPU acceleration. Is there something else out there or in development that would help here?
- mtw 11y ago
  the AI group at Facebook is using AWS instances with GPU. When you do the maths, it's quite interesting for research
  - PascLeRasc 11y ago
    late reply, but is that fairly simple to add to a workflow for people not completely immersed in tech?

58 points by Katydid 11y ago

We use CST microwave studio (https://www.cst.com/Products/CSTMWS https://www.cst.com/Products/CSTMWS) for a project. Calculation time on a Xeon E5-2680 was about 50-60 hours per simulation, together with a Tesla K80 it was reduced to about 4-5 hours. Pretty impressive. Adoption rate isn't very high, since most applications still don't use that power.

fmeyer 11y ago

NVidia gave me a Titan X for my learning experiments, and after all these years, I'm really excited about finish building a PC again.

jawilson2 11y ago
NVidia has been great for this. Back in 2009 they gave me 2 Teslas (C2060) and 2 Quadro FX 5800 cards. Then, I was able to get some results from those to get an NVidia Professor Partnership grant of $25000 (as a post doc). They had a large part in launching my engineering career doing HPC.

danieldk 11y ago

nVidia's Tesla processors and high-end GPUs are really stellar. We recently bought a machine with a Tesla K20c with some colleagues and the speed-up for neural net training has been tremendous. E.g. for some of my nets (NLP tasks) training time has gone from hours to 15-30 minutes. This is great when you are just experimenting with different approaches to a problem and want to iterate quickly. It's also great that nVidia hasn't been sitting on their laurels and released a library (cuDNN) of NN primitives that nearly everyone has been rebasing their packages on.

ihsw 11y ago

Why does NVidia dominate GPU computing? Where is AMD in this space? I was under the impression that AMD's strongest R&D in the past few years has been directed at hybrid consumer desktops, surely this should translate directly into their server processing products.

wyldfire 11y ago
IMO it's all due to the developer ecosystem. NVIDIA had a big head start with CUDA. AMD effectively embraced CUDA and transformed the best of CUDA into OpenCL (jointly with NVIDIA, Apple, via Khronos). NVIDIA's OpenCL support lags the standard, while they keep improving CUDA. NVIDIA probably feels commitment to their early adopters and doesn't want to divide up their engineering capability. Meanwhile NVIDIA also seems to have really good marketing behind their GPU software stack. Lots of people who use CUDA have no idea that OpenCL exists and no idea that porting between CUDA and OCL is not super difficult. (It's been years since I've looked at CUDA it might have diverged more significantly since). AMD's stability (on linux at least) is pretty poor. I don't know if it hurts adoption much, but it can't help. Also, AMD absolutely dominates on integer workloads which is why it had been used so much for cryptocoin hashing machines prior to the widespread availability of ASICs for most algorithms. On floating point workloads, NVIDIA might do better than AMD, and those seem much more common in science than integer math.
- nhaehnle 11y ago
  > AMD's stability (on linux at least) is pretty poor. Have you had any concrete issues recently? This may be one of these things where public perception changes slowly, unfortunately.
  - wyldfire 11y ago
    Yes, I have. :( I believe that they may be improving, but in general I have seen far fewer functionality problems from NVIDIA than AMD. Generation after generation, it seems to replay itself.
    - nhaehnle 11y ago
      Can you give details? If it's with the open driver stack, I may be able to help. Contact me at username at gmail if you like.
  - bainsfather 11y ago

sorenjan 11y ago

AMD isn't even mentioned in this article, and I recently read that nobody uses AMD for machine learning. Is this only down to the proprietary Cuda having more momentum than OpenCL? I've used both, albeit not very much, and my impression was that Cuda was a bit easier to write but OpenCL didn't have any deal breakers. I know AMD cards were the most popular for crypto currencies, so it's not like they're always the slowest. Granted, I only have experience with each company's consumer cards, and I think Nvidia holds their GeForce cards back more than AMD does their Radeon cards concerning compute performance. Why is Nvidia this dominant, and why are so many researchers using a proprietary API like Cuda when there's open alternatives? Related to this, are anybody using Intel's Xeon Phi? Tianhe-2 [0], the worlds fastest supercomputer since June 2013 use them as co-processors, but I rarely if ever hear about them in other projects. [0] https://en.wikipedia.org/wiki/Tianhe-2 https://en.wikipedia.org/wiki/Tianhe-2

DHJSH 11y ago
We find the price performance of NVIDIA hard to beat, combined with excellent compiler and developer support. We get better performance writing CUDA code directly than OpenCL, and it's easier for us to develop and debug. Yes, it's "proprietary" but we need to get our job done.
- sorenjan 11y ago
  It's in Nvidia's best interest to have their own Cuda be faster than OpenCL. Question is, which is faster: OpenCL on AMD, or Cuda on Nvidia?
lordCarbonFiber 11y ago
I have a little experience in GPU programming so I can address your first question. ;TLDR It is mostly due to momentum. The difference in momentum stems from a difference in design paradigms. OpenCL is just a library that you can link to can compile using GCC. Because the code you're writing is C++ syntactically you're often jumping through a lot more hoops. Nvidia uses their own compiler, so when you're writing cuda programs you have something that looks a lot like C++, however they have the freedom to extend the syntax to ease development. The marketing arm of Nvidia, combined with the ease of use helped get Cuda the defacto research platform. Once that ball was rolling, it becomes easier to find cuda teachers and cuda examples so it's naturally become more prolific.

marssaxman 11y ago

I've been ignoring GPUs for years, waiting for open standards, public documentation, and open-source drivers. I'm really disappointed that so much work is still being poured into these mysterious proprietary architectures. We wouldn't tolerate this anywhere else; why are GPUs allowed to be an exception?

jjaredsimpson 11y ago
Because ideology doesn't get work done. This is an example of making the perfect the enemy of the good. I assume you've been ignoring GPUs for years because you don't work in a problem space where you can leverage their power. Lots of other people however are in those spaces getting work done.
- marssaxman 11y ago
  I actually used to do a fair bit of work involving embarrassingly parallel algorithms on floating-point numbers - audio & video synthesis stuff. My last project was a little language/compiler system built around vectorization. I stopped working on these problems after it became clear that GPUs were now the best way to accomplish them. I thought I'd wait a year or two and get back into it after the relevant documentation came out. It's been over a decade and I'm still waiting. Fortunately, there are plenty of other interesting problems to work on. Ideology may not get work done, but secrecy also prevents work from getting done. I can't continue with the line of work I was pursuing back then as long as the relevant interfaces are wrapped up in a binary blob.

PascLeRasc 11y ago

It seems to me that there would be a massive demand to be able to work entirely in OS X by researchers for whom computing is a secondary task (I've seen this a lot in my neurobiology department). Most of these people are only knee-deep in the computing world, and learning Linux is not worth it to them. I know several researchers just getting into machine learning for their image processing/CNN projects, and they're just living with the speed of a 3+ year old Mac Pro, though they have the funding and desire to add plenty of GPU acceleration. Is there something else out there or in development that would help here?

mtw 11y ago
the AI group at Facebook is using AWS instances with GPU. When you do the maths, it's quite interesting for research
- PascLeRasc 11y ago
  late reply, but is that fairly simple to add to a workflow for people not completely immersed in tech?